Pith. sign in

REVIEW 4 major objections 6 minor 35 references

Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a 32B-parameter chatbot, trained with an LLM critic that scores and corrects responses, outperforms larger universal LLMs and existing proactive-dialogue methods on user-oriented proactivity.

desk verdict A useful dataset and a plausible training recipe, but the headline numbers measure how well the model satisfies the same judge that filtered its training data. read the letter →

arxiv 2505.12334 v1 pith:JDBY6SUA submitted 2025-05-18 cs.AI

Yufeng Wang , Jinwu Hu , Ziteng Huang , Kunyang Lin , Zitian Zhang , Peihao Chen , Yu Hu , Qianyue Wang
show 5 more authors
Zhuliang Yu Bin Sun Xiaofen Xing Qingfang Zheng Mingkui Tan
This is my paper · ORCID
classification cs.AI
keywords user-orientedproactivityopen-domaindialoguecritic-guidedcorpusgenerationLLM-as-a-judgecurriculumlearninguserbackgrounddatasetISCO-800system
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that open-domain chatbots fall short not because they cannot hold a conversation but because they do not proactively learn who the user is and steer the dialogue toward the user's own interests and background. To fix this, the authors build a User-oriented Proactive Chatbot (UPC) trained without human-written proactive dialogue data: an LLM critic, prompted as the user, scores each response on background relevance, user interest, and response value, and weak responses are regenerated from the critic's written feedback until they pass. The resulting high-quality dialogues are used to fine-tune the chatbot, and training proceeds in an easy-to-hard curriculum across 800 diverse user personas (ISCO-800) so the model gradually adapts to harder users. The paper reports that a 32B-parameter UPC outperforms larger universal LLMs, target-oriented proactive methods, and other LLM-based dialogue systems on relevance, interest, value, and perplexity, and that 62 real users preferred it over a 70B baseline more than 75% of the time. If these results hold, user-oriented proactivity can be instilled in any open-domain LLM cheaply, using a judge model as the data engine.

What carries the argument

The load-bearing mechanism is the critic-guided regeneration loop: an LLM judge role-playing the user scores each candidate response on background relevance, interest, and value, and its written criticism is fed back to the chatbot so the response is regenerated until it satisfies the judge. This loop converts an ordinary base model into a source of its own high-quality proactive training data, removing the need for human-annotated proactive dialogues. Three components support it: the ISCO-800 dataset of 800 persona descriptions built from 40 ISCO-08 occupational groups; the difficulty measurer, which labels a user's dialogue as easy only if every scored facet stays above a threshold and enough facets improve under regeneration; and the iterative curriculum scheduler, which fine-tunes on easy users first and admits harder users as the model improves.

What would settle it

Collect a held-out set of chatbot responses from UPC and a strong baseline, have independent human raters score them on the same relevance, interest, and value rubrics, and check the rank correlation with the LLM critic's scores. The central claim collapses if the critic's ordering of responses agrees no better than chance with human ordering, or if human preference for UPC over the baseline is absent in a larger preregistered study.

Watch

Extended reading notes

Core claim

The central claim is that user-oriented proactivity—the chatbot's ability to discover the user's background and evolving interests and lead the conversation onto user-centered topics—can be built into an open-domain chatbot through a closed loop of LLM judging and regeneration. The paper defines the target behavior as three scoreable facets: background relevance, user interest, and response value. A critic prompted with the user's persona and dialogue history rates each chatbot turn from 1 to 5 on these facets; turns scoring below 4 on any facet are sent back to the chatbot with the critic's reasons and regenerated until they clear the bar or the retry budget runs out. The filtered corpus fine-tunes the chatbot, and the loop repeats across iterations while a difficulty measurer admits only users whose dialogues are easy for the current model, gradually widening to harder users. On the ISCO-800 test set the resulting 32B-parameter UPC beats all compared methods on all reported metrics, and a 62-participant blind comparison finds that more than three quarters of users prefer it to Llama-3-70B-Instruct.

Load-bearing premise

The paper's load-bearing premise is that the LLM critic, when given a user's background and chat history, scores responses the way the real user would, and this same critic both filters the training data and computes the headline evaluation numbers; if the judge's scores are biased or gameable, the measured gains may not show up with real users.

Editorial extensions

If this is right

  • A 32B-parameter model can beat 70B and larger models on user-focused conversational quality, so raw parameter count is not the main lever for proactivity.
  • The critic-guided loop is model-agnostic: the paper reports consistent gains on Qwen1.5-14B, Qwen1.5-32B, and Vicuna-33B, with larger gains on stronger base models.
  • Training no longer depends on a human-annotated proactive dialogue corpus; the bottleneck shifts to judge reliability and persona diversity.
  • Curriculum order matters for efficiency as well as final quality: the regeneration rate falls from about 37.7% to 23.2% over four iterations as the model learns easier users first.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the judge's scores are trustworthy, the same critic-guided regeneration loop could be applied to task-oriented assistants (recommendation, tutoring, customer support) where personalization is also judged through a role-played user.
  • ISCO-800 is a reusable benchmark asset: future chatbots can be tested for user-oriented proactivity against the same 800 personas, making results across systems directly comparable.
  • A natural stress test the paper does not run is replacing the proprietary judge with a smaller open-weights critic during training, which would show whether the gains depend on judge quality or on the loop itself.
  • Because the same critic filters the training data and computes the headline metrics, part of the reported gain could be judge self-consistency; an independent human-rated benchmark would separate genuine proactivity improvements from judge artifacts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes UPC (User-oriented Proactive Chatbot), a training framework for open-domain dialogue that enhances user-oriented proactivity. The method has three components: an LLM-based critic that scores chatbot responses on relevance to user background, user interest, and response value; a critic-guided dialogue corpus generation procedure in which low-scoring responses are regenerated until the critic is satisfied; and a communication-difficulty-aware iterative curriculum that trains the chatbot first on easy-to-serve users from a new ISCO-800 dataset. Experiments on ISCO-800 report that a 32B-parameter Qwen1.5 model fine-tuned with UPC outperforms larger baselines on the critic-based metrics and on perplexity. A small online study with 62 participants compares UPC with Llama-3-70B-Instruct and reports that over 75% of participants preferred UPC on overall quality and on each of the three dimensions.

Significance. The paper addresses a real and underexplored problem: making open-domain chatbots proactively attend to the user's background and interests rather than merely following the user or steering to system-chosen targets. The ISCO-800 dataset is a potentially useful resource, and the idea of using critic-guided regeneration together with a communication-difficulty curriculum is plausible and reasonably motivated. The authors also provide ablations, apply the method to multiple base LLMs, and include a real-user pilot, which is more than many dialogue papers do. However, the headline claim that UPC 'outperforms all other methods' is not currently established because the central evaluation metric is the same class of LLM judge that filtered the training data, and the sole external human study is too small and too narrow to carry the claim. If the authors can close this gap with a rigorous human evaluation or a convincing critic-human agreement analysis, the paper could be a solid contribution.

major comments (4)
  1. [§4.2, Algorithm 1 vs. §5.1–5.2]
  2. [§5.4, Figure 5]
  3. [§5.1, Table 1]
  4. [§4.3, Eq. (3) and Appendix B]
minor comments (6)
  1. [Table 2]
  2. [Figure 5]
  3. [Algorithm 1, line 12]
  4. [§5.1, Compared Methods]
  5. [Algorithm 1]
  6. [Abstract and Contributions]

Circularity Check

1 steps flagged · score 6.0 of 10

Headline Rel./Int./Val. gains are scored by the same critic that filtered and selected the training corpus, so Table 1 largely measures critic alignment rather than independent user experience.

  1. fitted input called prediction [Algorithm 1 (lines 12-15), Section 4.2; Section 5.1 Metrics; Table 1]
    "while(𝑆𝑖,𝑡𝑖𝑛𝑡. ,𝑆𝑖,𝑡𝑟𝑒𝑙. or𝑆𝑖,𝑡𝑟𝑒𝑙. < 4)𝑎𝑛𝑑𝑟 <=𝑅 do ... Re-generate utterance 𝑢𝐵𝑜𝑡𝑖,𝑡 ,𝑟 =𝑟+ 1. ... Compute scores S𝑖,𝑡 for𝑢𝐵𝑜𝑡𝑖,𝑡 via Eqn. (1). ... We use our designed scoring system (including the background relevance𝑅𝑒𝑙. , user’s interest level𝐼𝑛𝑡. , and response value 𝑉𝑎𝑙. ) to evaluate the chatbot’s user-oriented proactivity. Notably, the critic used for evaluation includes gpt-3.5-turbo-0125 or gpt-4-turbo-2024-04-09."

    Equation (1) defines the three reported metrics as the output of the critic J, and Algorithm 1 uses exactly this J as a rejection-sampling filter: any chatbot response with a dimension below 4 is regenerated until the critic approves it, and only the accepted high-score corpus is used for fine-tuning. Section 5.1 then evaluates the trained model and all baselines with the same “designed scoring system” using gpt-3.5-turbo-0125 or gpt-4-turbo. The headline Table 1 numbers therefore measure how well UPC satisfies the very judge that curated its training data. The GPT-4 evaluator uses the same prompt and rubric, and the paper itself says GPT-4 shows “similar scoring trends” to the GPT-3.5 training judge, so it is not an independent yardstick.

full rationale

The paper's central claim is that UPC outperforms all baselines on Rel., Int., Val., and PPL (Section 5.2, Table 1). However, Rel., Int., and Val. are not external metrics: they are computed by the LLM critic J defined in Eq. (1). The same critic drives training: Algorithm 1 regenerates every response until the critic gives scores >= 4 on all dimensions, and the curriculum learning step (Algorithm 2, Eq. (3)) selects 'easy' users based on those same critic scores. Thus the training pipeline is explicitly optimizing the responses to maximize the evaluation metric. Reporting the post-optimization critic scores as 'outperforms all other methods' is therefore a claim about how well the model matches the critic's preferences, not about genuine user-oriented proactivity. The GPT-4 evaluator shares the identical prompt and rubric and is only described as having 'similar scoring trends', so it does not break the loop. The human study is the sole external evidence, but with 62 participants, no significance tests, and only one baseline (Llama-3-70B), it cannot support the 'all other methods' claim. The paper is not circular in the sense of a logical tautology (the trained model does produce new responses and the critic does provide some signal), and there is no self-citation chain of concern; but the headline quantitative claim reduces by construction to the same LLM judge used to train the model. PPL is a language-model perplexity and does not measure proactivity, so it does not provide independent validation. Overall, this is a partial circularity: the reported superiority on the primary metrics is partly a fit to the evaluating critic.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method rests on the validity of the LLM critic as a human-aligned evaluator, the realism of simulated user agents, and the correctness of the difficulty measure and curriculum. Several hyperparameters (alpha, beta, regeneration threshold) are hand-chosen and not fully specified in the main text.

free parameters (6)
  • alpha
    In Eq. (3), a sample is 'difficult' if any metric is below alpha. Value not specified in main text, likely tuned on validation.
  • beta
    In Eq. (3), threshold for the number of boosted metrics that must be present to consider a sample easy. Value not specified in main text.
  • regeneration_threshold = 4
    Responses are regenerated until all critic scores are 4 or higher, a hand-chosen threshold that directly shapes the training corpus.
  • max_regeneration_attempts
    Maximum number of regeneration attempts R in Algorithm 1, not specified in main text.
  • dialogue_turns = 5
    Each dialogue consists of 5 turns, a design choice for the experiments.
  • iterations = 4
    Training converges at 4 iterations, as stated in Section 5.4.
assumptions (5)
  • domain assumption An LLM-as-a-judge critic can accurately and reliably score user-oriented proactivity from a user background and dialogue history.
    Section 4.1 introduces this critic and uses it both to guide training and to evaluate results; if this fails, the method's core loop is invalid.
  • domain assumption A Qwen1.5-72B-Chat user agent simulating a persona faithfully reproduces real user behavior and preferences.
    Section 4.2 generates all training data by chatting with these simulated users; transfer to real users is assumed.
  • domain assumption The communication difficulty measure in Eq. (2)-(3) correctly ranks users by how hard they are for the chatbot to engage.
    This measure drives the curriculum scheduler, so its validity determines whether curriculum learning helps.
  • domain assumption Regenerating low-scoring responses with critic feedback improves the collected corpus more than alternatives.
    Core of the critic-guided data generation paradigm; the paper provides no comparison to other data-improvement methods.
  • domain assumption ISCO-800 generation prompt produces a diverse and representative sample of user backgrounds.
    Section 4.4 uses GPT-4 with a prompt asking for 20 varied backgrounds per occupation group; representativeness is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance." pith.science (2026). https://pith.science/paper/JDBY6SUA

@misc{pith2026250512334,
  author       = {Pith},
  title        = {Pith review of: Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JDBY6SUA}},
  note         = {Machine review of arXiv:2505.12334}
}
read the original abstract

Open-domain dialogue systems aim to generate natural and engaging conversations, providing significant practical value in real applications such as social robotics and personal assistants. The advent of large language models (LLMs) has greatly advanced this field by improving context understanding and conversational fluency. However, existing LLM-based dialogue systems often fall short in proactively understanding the user's chatting preferences and guiding conversations toward user-centered topics. This lack of user-oriented proactivity can lead users to feel unappreciated, reducing their satisfaction and willingness to continue the conversation in human-computer interactions. To address this issue, we propose a User-oriented Proactive Chatbot (UPC) to enhance the user-oriented proactivity. Specifically, we first construct a critic to evaluate this proactivity inspired by the LLM-as-a-judge strategy. Given the scarcity of high-quality training data, we then employ the critic to guide dialogues between the chatbot and user agents, generating a corpus with enhanced user-oriented proactivity. To ensure the diversity of the user backgrounds, we introduce the ISCO-800, a diverse user background dataset for constructing user agents. Moreover, considering the communication difficulty varies among users, we propose an iterative curriculum learning method that trains the chatbot from easy-to-communicate users to more challenging ones, thereby gradually enhancing its performance. Experiments demonstrate that our proposed training method is applicable to different LLMs, improving user-oriented proactivity and attractiveness in open-domain dialogues.

Figures

Figures reproduced from arXiv: 2505.12334 by the authors.

Figure 1
Figure 1. Illustration of different open-domain dialogue methods. Left: universal LLMs. Medium: target-oriented methods. Right: our [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. General diagram of proposed UPC. We train UPC in an iterative process. In each iteration, the chatbot engages in dialogue with user [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The prompt of the critic. 4.1 Evaluation of User-oriented Proactivity To evaluate the user-oriented proactivity S𝑖,𝑡 from the three perspectives of user’s interest level, background rele￾vance, and response value, we follow the LLM-as-a-judge method [Zheng et al., 2024] and construct the critic J by prompting the ChatGPT to score the abilities. Specifically, we let the critic obtain the user’s background and the dia… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The regeneration prompt for the chatbot. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The performance during the iterative curriculum learning. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Case study of our UPC compared with GPT-3.5-Turbo. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages

  1. [1]

    Gpt-4 technical report

    [Achiam et al., 2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Alt- man, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv 2023

  2. [4]

    Towards robust personalized dialogue generation via order-insensitive representation regularization

    [Chen et al., 2023] Liang Chen, Hongru Wang, Yang Deng, Wai Chung Kwan, Zezhong Wang, and Kam-Fai Wong. Towards robust personalized dialogue generation via order-insensitive representation regularization. In Find- ings of the Association for Computational Linguistics: ACL 2023, pages 7337–7345,

    work page 2023

  3. [5]

    Recent trends in per- sonalized dialogue generation: A review of datasets, methodologies, and evaluations

    [Chen et al., 2024] Yi-Pei Chen, Noriki Nishida, Hideki Nakayama, and Yuji Matsumoto. Recent trends in per- sonalized dialogue generation: A review of datasets, methodologies, and evaluations. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC- COLING 2024), pages 13650–13665,

    work page 2024

  4. [6]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    [Chiang et al., 2023] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6,

    work page 2023

  5. [7]

    Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non- collaboration

    [Deng et al., 2023c] Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, and Tat-Seng Chua. Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non- collaboration. In Findings of the Association for Compu- tational Linguistics: EMNLP 2023 , pages 10602–10621,

    work page 2023

  6. [8]

    Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum

    [Gao et al., 2024] Shen Gao, Zhengliang Shi, Minghang Zhu, Bowen Fang, Xin Xin, Pengjie Ren, Zhumin Chen, Jun Ma, and Zhaochun Ren. Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, volume 38, pages 18030–18038,

    work page 2024

  7. [9]

    Knowledge-grounded dialogue flow management for social robots and conver- sational agents

    [Grassi et al., 2022] Lucrezia Grassi, Carmine Tommaso Recchiuto, and Antonio Sgorbissa. Knowledge-grounded dialogue flow management for social robots and conver- sational agents. International Journal of Social Robotics, 14(5):1273–1293,

    work page 2022

  8. [11]

    Dynamic compressing prompts for efficient inference of large lan- guage models

    [Hu et al., 2025] Jinwu Hu, Wei Zhang, Yufeng Wang, Yu Hu, Bin Xiao, Mingkui Tan, and Qing Du. Dynamic compressing prompts for efficient inference of large lan- guage models. arXiv preprint arXiv:2504.11004,

    arXiv 2025

Show all 35 references
  1. [14]

    Proactive conversational agents in the post-chatgpt world

    [Liao et al., 2023] Lizi Liao, Grace Hui Yang, and Chirag Shah. Proactive conversational agents in the post-chatgpt world. In Proceedings of the 46th International ACM SI- GIR Conference on Research and Development in Infor- mation Retrieval, pages 3452–3455,

    2023

  2. [15]

    Interprofessional com- munication and its challenges

    [Lindqvist, 2015] Susanne Lindqvist. Interprofessional com- munication and its challenges. Clinical communication in medicine, pages 157–167,

    2015

  3. [17]

    [Lu et al., 2023] Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu

    https: //openlm.ai/chatbot-arena/. [Lu et al., 2023] Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. Memochat: Tuning llms to use memos for consis- tent long-range open-domain conversation. arXiv preprint arXiv:2308.08239,

    2023 arXiv

  4. [18]

    A critical review of state-of-the- art chatbot designs and applications

    [Luo et al., 2022] Bei Luo, Raymond YK Lau, Chunping Li, and Yain-Whar Si. A critical review of state-of-the- art chatbot designs and applications. Wiley Interdisci- plinary Reviews: Data Mining and Knowledge Discovery, 12(1):e1434,

    2022

  5. [19]

    Engagement in human-agent interaction: An overview

    [Oertel et al., 2020] Catharine Oertel, Ginevra Castellano, Mohamed Chetouani, Jauwairia Nasir, Mohammad Obaid, Catherine Pelachaud, and Christopher Peters. Engagement in human-agent interaction: An overview. Frontiers in Robotics and AI, 7:92,

    2020

  6. [21]

    com/blog/chatgpt/

    https://openai. com/blog/chatgpt/. [OpenAI, 2024] OpenAI. GPT-4o System Card,

    2024

  7. [22]

    [Organization, 2008] International Labour Organization

    https: //openai.com/index/gpt-4o-system-card/. [Organization, 2008] International Labour Organization. In- ternational Standard Classification of Occupations 2008,

    2008

  8. [24]

    Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage

    [Shuster et al., 2022] Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2...

    2022 arXiv

  9. [25]

    Towards diverse, relevant and coherent open-domain dialogue generation via hybrid latent variables

    [Sun et al., 2023] Bin Sun, Yitong Li, Fei Mi, Weichao Wang, Yiwei Li, and Kan Li. Towards diverse, relevant and coherent open-domain dialogue generation via hybrid latent variables. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13600–13608,

    2023

  10. [26]

    Target-guided open-domain conversation

    [Tang et al., 2019] Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric Xing, and Zhiting Hu. Target-guided open-domain conversation. In Pro- ceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5624–5634,

    2019

  11. [28]

    A survey on curriculum learning

    [Wang et al., 2021] Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence, 44(9):4555– 4576,

    2021

  12. [30]

    Learning by trial and error

    [Young, 2009] H Peyton Young. Learning by trial and error. Games and economic behavior, 65(2):626–643,

    2009

  13. [32]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36,

    [Zheng et al., 2024] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36,

    2024

  14. [33]

    [ZHIPUAI, 2024] ZHIPUAI. GLM-4,

    2024

  15. [34]

    https://chatglm. cn/. [Zhong et al., 2024] Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19724–19731,

    2024

  16. [35]

    Lima: Less is more for align- ment

    [Zhou et al., 2024] Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for align- ment. Advances in Neural Information Processing Sys- tems, 36, 2024

    2024

  17. [1977]

    Open-domain dialogue generation: What we can do, cannot do, and should do next

    [Kann et al., 2022] Katharina Kann, Abteen Ebrahimi, Joewie Koh, Shiran Dudy, and Alessandro Roncone. Open-domain dialogue generation: What we can do, cannot do, and should do next. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 148–165,

    2022

  18. [2008]

    [Rodrigues et al., 2022] Carlos Rodrigues, Ars ´enio Reis, Rodrigo Pereira, Paulo Martins, Jos ´e Sousa, and Tiago Pinto

    www.ilo.org/publications/international-standard- classification-occupations-2008-isco-08-structure. [Rodrigues et al., 2022] Carlos Rodrigues, Ars ´enio Reis, Rodrigo Pereira, Paulo Martins, Jos ´e Sousa, and Tiago Pinto. A review of conversational agents in education. In Inte...

    2008

  19. [2009]

    A survey of large language models

    [Zhao et al., 2023] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2),

    2023 arXiv

  20. [2015]

    Chatbot Arena,

    [LMSYS, 2024] LMSYS. Chatbot Arena,

    2024

  21. [2019]

    Characterchat: Learning towards conversa- tional ai with personalized social support

    [Tu et al., 2023] Quan Tu, Chuanqi Chen, Jinpeng Li, Yan- ran Li, Shuo Shang, Dongyan Zhao, Ran Wang, and Rui Yan. Characterchat: Learning towards conversa- tional ai with personalized social support. arXiv preprint arXiv:2308.10278,

    2023 arXiv

  22. [2020]

    ChatGPT,

    [OpenAI, 2022] OpenAI. ChatGPT,

    2022

  23. [2021]

    Target-constrained bidirectional planning for genera- tion of target-oriented proactive dialogue

    [Wang et al., 2024] Jian Wang, Dongding Lin, and Wenjie Li. Target-constrained bidirectional planning for genera- tion of target-oriented proactive dialogue. ACM Transac- tions on Information Systems, 42(5):1–27,

    2024

  24. [2022]

    Dynamic ensemble reasoning for llm experts

    [Hu et al., 2024] Jinwu Hu, Yufeng Wang, Shuhai Zhang, Kai Zhou, Guohao Chen, Yu Hu, Bin Xiao, and Mingkui Tan. Dynamic ensemble reasoning for llm experts. arXiv preprint arXiv:2412.07448,

    2024

  25. [2023]

    Llama 3 model card

    [AI@Meta, 2024] AI@Meta. Llama 3 model card

    2024

  26. [2024]

    Qwen technical report

    [Bai et al., 2023] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xu- ancheng...

    2023 arXiv

  27. [2025]

    Perplexity—a measure of the difficulty of speech recognition tasks

    [Jelinek et al., 1977] Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. Perplexity—a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S63–S63,

    1977

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.