Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

Changhe Song; Chunpu Xu; Dongsheng Li; Jiashuo Wang; Kaitao Song; Lili Qiu; Wenjie Li; Yang Xiao

REVIEW 5 major objections 6 minor 37 references

Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that aligning interactive LLMs with DPO on preference pairs gathered by an interaction-aware Monte Carlo tree search raises user engagement, lifting the engaged rate from 64.06% to 80.47% in emotional support and doubling…

desk verdict The i×MCTS + DPO pipeline is genuinely new and its preference data is human-validated, but the headline automated gains in Table 2 are simulator self-consistency rather than demonstrated real-user engagement. read the letter →

arxiv 2506.21497 v1 pith:2ZLTSQVF submitted 2025-06-26 cs.CL

Jiashuo Wang , Kaitao Song , Chunpu Xu , Changhe Song , Yang Xiao , Dongsheng Li , Lili Qiu , Wenjie Li This is my paper

classification cs.CL

keywords userengagementinteractiveLLMdirectpreferenceoptimizationMonteCarlotreesearchsimulatoremotionalsupportdialoguepersuasionforgoodsocially-driven

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an interactive large language model can be made more engaging by aligning it with direct preference optimization on response pairs that a tree search found to lead to better or worse conversational outcomes. The key move is to reward the model for what happens later in the conversation, not for sounding knowledgeable or following a dialogue strategy: whether the simulated user eventually expresses all feelings, or decides to donate more. In automated interactive evaluation, the aligned supporter raises the engaged rate from 64.06% to 80.47% in emotional support, and the aligned persuader doubles the average donation from $0.58 to $1.29, while conversation length stays roughly constant. The paper argues that engagement can be learned from the future development of a conversation rather than from per-turn proxies.

What carries the argument

The central object is i×MCTS, a Monte Carlo tree search whose nodes alternate between model utterances and user-simulator utterances. Terminal rewards come from rule-based engagement detection: regular-expression matching for whether the simulated help-seeker's final state still contains negative feelings, and extraction of the decided donation amount for the persuadee. The search produces preference pairs that are converted into a DPO training signal, with UCB selection guiding exploration and similarity-based pruning keeping the tree manageable.

What would settle it

Run a real-user study on both tasks comparing the SFT and Aligned models on terminal engagement rates and donation amounts; if the engaged-rate gap (64.06% to 80.47%) and the donation gap ($0.58 to $1.29) collapse, the central claim is falsified. A complementary calibration check is to have annotators distinguish simulator terminal states from real user terminal states in a forced-choice test; near-chance discrimination would show the simulator does not model real engagement.

Watch

Extended reading notes

Core claim

The paper introduces i×MCTS (Monte Carlo Tree Search for interaction), a search over simulated user–model conversations in which the reward is the user's terminal reaction rather than any per-turn quality proxy. The tree alternates model-generated responses with utterances from a user simulator trained by supervised fine-tuning, and back-propagates a reward of 1 only when a trajectory ends with an engagement signal. From the resulting search tree the method extracts chosen and rejected model responses to the same context, supplements them with reward-model-ranked pairs from the training set, and fine-tunes the interactive LLM with DPO. The paper reports that the aligned model improves engagement in both tested scenarios and that its human ratings and small user study agree with the automated evaluation.

Load-bearing premise

The load-bearing premise is that the user simulator, trained by supervised fine-tuning on the same corpora, responds like real users would, and that the terminal signals (no remaining negative feelings, donation amount) capture engagement; if the simulator diverges from real users, the preference data and evaluation rewards encode an artificial target.

Editorial extensions

If this is right

User engagement can be treated as a terminal reward and learned from future conversation outcomes, removing the need for per-turn engagement labels.
The same pipeline transfers across two scenarios that define engagement differently, suggesting the recipe generalizes to other goal-oriented social dialogues.
The aligned model holds conversation length nearly constant while improving outcomes, so the engagement gains are not simply an artifact of longer interactions.
Explicit strategy or knowledge rewards offer smaller gains than the future-outcome reward, indicating that steering toward the user's end state matters more than planning content alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to replace the fixed regex and sentiment detectors with learned classifiers of terminal engagement, which may transfer better when real users replace the simulator.
If the simulator is a reliable proxy, the same outcome-reward recipe could apply to other dialogues with an observable terminal state, such as tutoring, health coaching, or negotiation.
Monitoring whether simulated gains persist with real users would test the method's practical value, since the paper's own calibration reported only 48% human identification of simulator outputs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know before you read arXiv:2506.21497. The method is a genuinely new combination: i×MCTS runs MCTS on a simulated conversation between the target LLM and a user simulator, uses end-of-conversation engagement signals (regex-detected absence of negative feelings in emotional support; donation amount in persuasion) as reward, and distills the resulting chosen/rejected pairs into the model with DPO. Prior MCTS dialogue work was about planning dialogue acts; here the search directly optimizes the interactive policy, which is a real difference. Second, read the headline numbers carefully: the same user simulator produces the reward in i×MCTS, the preference dataset, and the automated evaluation in Table 2. The 80.47% engaged rate and $1.29 donation are measures of fit to that simulator, not demonstrated real-user engagement.

Credit where it is due. The preference pairs are human-checked: Fleiss kappa of 0.67 and 0.59 between two annotators and the i×MCTS ranking on 160 pairs per task, with disagreements mostly ties. That is decent evidence the training signal is sensible. Adding Dt to avoid catastrophic forgetting is a reasonable safeguard. The human evaluations are small but blinded and point the expected way.

Soft spots, in order. The closed loop is the main one. Appendix A.3 reports the simulator calibration honestly: over 50 pairs, annotators labeled 48% human, 32% simulator, 20% unsure. That is not strong evidence the simulator tracks real users, and it does not support the transfer claims. Table 5 contradicts Figure 3: the SFT and Aligned rows in Table 5 appear swapped, with SFT assigned the scores Figure 3 gives Aligned. Probably a labeling error, but it needs fixing before anything else. Table 2 has no confidence intervals or significance tests. The human eval is small: 3 RAs, 36 conversations; user study 5 participants, 19 conversations.

My read: the central idea is sound and the paper is coherent on its own terms. The i×MCTS pipeline is a legitimate contribution to simulator-based dialogue alignment, and the authors disclose their own weak calibration rather than hiding it. Missing: evidence that simulator gains transfer to real users, or claims limited to what the simulator can show.

For: people working on engagement in dialogue, DPO for conversational models, or user-simulator evaluation. Deserves a serious referee. Recommendation: send to peer review, expect major revision — fix Table 5, report uncertainty on Table 2, release code and preference data, and either validate the simulator against real users or temper the claims.

Referee Report

5 major / 6 minor

Summary. The paper proposes i×MCTS, a Monte Carlo Tree Search variant that interacts with a trained user simulator, to collect preference pairs over LLM responses; the LLM is then aligned with DPO using these preferences, where the reward is an end-of-conversation engagement signal (regex-detected absence of negative feelings in emotional support, donation amount in persuasion). Experiments on two socially-driven dialogue tasks report that the aligned model raises the engaged rate from 64.06% to 80.47% and doubles the donation amount from $0.58 to $1.29 while keeping conversation length roughly constant, and a small human evaluation is reported as consistent with these gains. The central claim is that direct alignment on future-dialogue outcome signals improves user engagement in interactive LLMs.

Significance. If the reported gains transfer to real users, the work makes a useful contribution: it shifts the optimization target from dialogue acts or knowledge to end-state engagement, and it provides a concrete pipeline (i×MCTS rollouts with a user simulator, preference extraction, DPO alignment) that others could adapt. The manuscript also has strengths in transparency: it provides an anonymous code link, reports reproducibility details in A.6, includes an inter-annotator agreement check for the preference dataset (Fleiss kappa 0.67 and 0.59 in the two tasks), and attempts both automated and human evaluation. However, the significance is conditional, because the automated evaluation is conducted with the same user simulator that generated the training preferences, and the human validation is too small and internally inconsistent to establish that the headline numbers reflect real-user engagement.

major comments (5)

[§5.3 vs. §3.1–§3.3] The headline automated results are measured against the same user simulator that produced the DPO training signal. Specifically, the engagement signal in §3.1 (regex absence of negative feelings; donation amount), the preference pairs in Dp from i×MCTS in §3.2–§3.3, and the evaluation in §5.3 all use the same simulator. The reported improvements from 64.06% to 80.47% and from $0.58 to $1.29 may therefore reflect the model learning simulator-specific regularities or regex-exploiting phrasing rather than genuine engagement. The paper needs an external validation step—for example, showing that the simulator-based preference signal correlates with real-user engagement outcomes—before the central claim is supported.
[Appendix A.3] The reliability check for the user simulator is too weak to break the closed loop. Over 50 conversation pairs, three annotators labeled outputs as human 48%, simulator 32%, and cannot-identify 20%. This is roughly chance-level discrimination and does not establish that the simulator behaves like real users; in fact, it suggests substantial uncertainty about simulator fidelity. The manuscript should provide a more discriminative calibration test, such as forced-choice human-vs-simulator identification with held-out real conversations, and ideally measure whether simulator-predicted engagement matches human-participant engagement on the same dialogue histories.
[Appendix A.5, Table 5 and Figure 3] Table 5 contradicts Figure 3 as printed. Figure 3 shows Aligned with higher mean supportiveness and willingness scores (8.14 and 8.53) and winning the majority of pairwise comparisons, while Table 5 assigns precisely those higher scores and non-overlapping confidence intervals to the SFT Supporter row, and the lower scores (6.94 and 7.14) to the Align Supporter row. This label swap undermines the human-rating claim in §5.4 unless corrected. The authors should verify the row labels and report the actual confidence intervals for each model consistently.
[§5.3, Table 2] Table 2 reports point estimates without confidence intervals or significance tests. The text states that the aligned model 'significantly increases' the engagement rate, but with 128 interactions and no repeated sampling (A.6 states only one sampling run per interaction was used), the claim of significance is unsupported. Bootstrap confidence intervals or a paired test over the same user conditions should be reported for both the engaged rate and the donation amount.
[§3.2, §4.5, Eq. (2)–(5)] The method contains several free parameters and heuristic modeling choices whose influence on the headline results is not examined: the UCB constant c in Eq. (2), expansion width e, max nodes K, max depth T, the sentiment discount factor γ in Eq. (5), the DPO temperature β in Eq. (3), and the pruning thresholds. In particular, the simplification in §3.2 of using a single user node per model node assumes away user-response diversity; this assumption is load-bearing for the search and should be tested, for instance by varying the number of user rollouts and measuring the change in preference quality or downstream engagement.

minor comments (6)

[§3.2] The word 'iteractively' appears in the description of the i×MCTS loop; this should be 'iteratively.'
[§4.6] The phrase 'In peruasion for good' contains a typo; it should be 'persuasion.'
[Table 3 caption] The caption says 'Helper-Seeker Simulator' in the SFT case in Table 6; for consistency, this should be 'Help-Seeker Simulator.'
[§5.2] The human ranking of the preference dataset is based on only two annotators; the authors should report how disagreements were resolved and whether the annotators were fluent in the Chinese used in the conversations.
[§5.1, BoN] The BoN baseline samples N=3 outputs, but the paper does not specify the sampling temperature or whether the reward model ranking is applied to each independently sampled response; this makes it hard to reproduce the BoN numbers.
[Appendix A.5, User Study] The user study with five participants and 19 conversations is very small, and the paper does not report per-participant statistics or the distribution of conversation lengths; a more detailed breakdown would help assess variability.

Circularity Check

3 steps flagged · score 8.0 of 10

The headline engagement gains are measured against the same user simulator that supplied the DPO preference signal; the evaluation metric is the training reward by construction, and the weak simulator calibration does not break the loop.

fitted input called prediction [Section 3.1 'Engagement'; Section 3.2 'Rollout'; Section 5.3 'Automated Interactive Evaluation'; Table 2]
"Therefore, we use regular expression matching to detect whether there is still any negative feelings and thoughts in the user simulator’s emotional & cognitive states in its output. ... If a user utterance indicates both conversation termination and engagement, the immediate reward is 1; otherwise, it is 0. ... We evaluate the performance of interactive LLM by their behavior in the interaction with user simulator ... We focus on the following two features: the user engaged rate (%) or donation amount ($)."

The same rule-based engagement signal used to label i×MCTS preference pairs and define the DPO training reward (Section 3.1-3.2) is the metric reported in Section 5.3 and Table 2. The DPO loss in Eq. 3 optimizes the policy to increase exactly this signal on the same user simulator that produces the final utterances; the automated evaluation then measures that same signal on that same simulator. The 64.06% to 80.47% engaged-rate gain and the $0.58 to $1.29 donation gain therefore report fit to the training judge, not an independent estimate of real-user engagement.
self definitional [Section 3.3 'Preference Dataset for Alignment']
"Dt, which contains response pairs generated by πθ given contexts from πθ’s training set and ranked by a reward model trained on Dp."

Dp is constructed from i×MCTS rollouts whose reward is the simulator's final engagement signal, and Dt inherits that signal through a reward model trained on Dp. Consequently all of D is generated from one operational definition of engagement: regex-detectable absence of negative feelings or a higher extracted donation amount in the simulator's last utterance. The 'engagement' the model is aligned to is defined, not measured independently, and the automated evaluation re-measures this same construct.

1 more flagged steps

self citation load bearing [Section 5.3, Automated Interactive Evaluation; Appendix A.3]
"We evaluate the performance of interactive LLM by their behavior in the interaction with user simulator, as in Wang et al. (2024a)."

The main evidence for engagement gains is the simulator-interaction evaluation, justified only by a citation to prior work by the same authors (Wang et al. 2024a). The paper's own calibration of that simulator (Appendix A.3: 48% human, 32% simulator, 20% cannot identify over 50 pairs) is weak, and the human checks are small and internally inconsistent (Table 5 contradicts Figure 3). Thus the self-citation supplies the evaluation loop without independent external validation; it does not break the training/evaluation circularity.

full rationale

The central derivation chain is: train a user simulator by SFT (Section 3.1); use it in i×MCTS with a final-state engagement reward (Section 3.2); build D_p and D_t from that signal (Section 3.3); align via DPO (Eq. 3); then report engagement as 'user engaged rate (%) or donation amount ($)' obtained from interactions with the same simulator (Section 5.3). The headline numbers in Table 2 therefore measure the model's ability to elicit the exact regex-defined final states that served as its training reward. This is a training/evaluation loop closed by construction, not an external falsification of the engagement claim. The paper's attempts to break the loop are insufficient: Appendix A.3 shows only 48% of simulator conversations were judged human, with 32% judged simulator and 20% uncertain; the human rating in Figure 3 is contradicted by its own reliability table (Table 5 lists overlapping CIs in the opposite order); and the user study has only 5 participants and 19 conversations. None of these provide the missing correlation between simulator-measured gains and real-user engagement. Because the central claim reduces to optimizing and then re-measuring the same simulator-based engagement signal, the paper is substantially circular, though not in the stronger sense of a mathematical theorem being presupposed. Score 8 reflects that the primary empirical result is forced by the shared definition and evaluation of engagement; it is not 10 because the method does contain independent components (MCTS construction, DPO, and a small human study) that could in principle support the claim if the simulator were properly validated.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The pipeline rests on the user simulator's fidelity, the choice of final-turn engagement signals, and several ad hoc pruning heuristics. These are domain assumptions and hand-chosen thresholds rather than standard math or pinned-down physical constants. No invented entities are introduced.

free parameters (8)

UCB exploration constant c = sqrt(2)
Set by hand in Eq. (2) for MCTS selection; standard choice, but influences tree exploration and preference pair quality.
Expansion width e = 3
Number of model node expansions per tree node (Section 4.6); affects coverage and compute.
Max nodes per depth K = 81
Pruning cap in Section 4.6; controls search breadth.
Max depth T = 25 and 15 for emotional support and persuasion
Search depth limits in Section 4.6; engagement signals only appear at conversation end.
Sentiment discount factor gamma = unspecified
Discount in Eq. (5) for accumulated sentiment during persuasion pruning; the value is not reported.
DPO temperature beta = unspecified
Beta in Eq. (3) for DPO loss; value is not given in the experimental settings.
Reward model on Dp = trained model, architecture unspecified
Used to rank Dt pairs (Section 3.3); introduces an additional learned component whose accuracy is not reported.
Engagement detection thresholds = regex rules for negative-feeling residue; donation normalized by $2
Defines the reward in Section 3.1; exact regexes are not released, and donation amount is assumed to measure engagement.

assumptions (6)

domain assumption User simulator trained via SFT faithfully represents real user behavior.
Section 3.1 and 4.4; the simulator generates both the interaction data for training and the evaluation outcomes. Appendix A.3's Turing-style test shows only 48% human detection, so fidelity is not established.
domain assumption Final-turn regex-detected signals are valid measures of user engagement.
Section 3.1; negative-feeling residue and donation amount are used as the only engagement indicators, without validation against multi-dimensional engagement constructs.
ad hoc to paper A single user node per model node captures user reaction diversity.
Section 3.2; this simplification removes stochasticity of user responses from the tree, potentially biasing engagement estimates.
ad hoc to paper Cosine similarity between simulator state and LLM predicted state selects supportive trajectories.
Section 4.5 Eq. (4); no evidence that state-agreement correlates with user engagement.
ad hoc to paper VADER sentiment accumulation predicts donation engagement.
Section 4.5 Eq. (5); sentiment proxy is used without calibration to donation decisions.
domain assumption Reward model trained on Dp ranks Dt pairs accurately.
Section 3.3; although Dp pairs have human agreement (Fleiss kappa 0.67 and 0.59), the reward model's accuracy on held-out pairs is not reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments." pith.science (2026). https://pith.science/paper/2ZLTSQVF

@misc{pith2026250621497,
  author       = {Pith},
  title        = {Pith review of: Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2ZLTSQVF}},
  note         = {Machine review of arXiv:2506.21497}
}

abstract

Enhancing user engagement through interactions plays an essential role in socially-driven dialogues. While prior works have optimized models to reason over relevant knowledge or plan a dialogue act flow, the relationship between user engagement and knowledge or dialogue acts is subtle and does not guarantee user engagement in socially-driven dialogues. To this end, we enable interactive LLMs to learn user engagement by leveraging signals from the future development of conversations. Specifically, we adopt a more direct and relevant indicator of user engagement, i.e., the user's reaction related to dialogue intention after the interaction, as a reward to align interactive LLMs. To achieve this, we develop a user simulator to interact with target interactive LLMs and explore interactions between the user and the interactive LLM system via \textit{i$\times$MCTS} (\textit{M}onte \textit{C}arlo \textit{T}ree \textit{S}earch for \textit{i}nteraction). In this way, we collect a dataset containing pairs of higher and lower-quality experiences using \textit{i$\times$MCTS}, and align interactive LLMs for high-level user engagement by direct preference optimization (DPO) accordingly. Experiments conducted on two socially-driven dialogue scenarios (emotional support conversations and persuasion for good) demonstrate that our method effectively enhances user engagement in interactive LLMs.

Figures

Figures reproduced from arXiv: 2506.21497 by the authors.

**Figure 1.** i×MCTS: We perform MCTS on a simulated conversation between the user simulator, which is provided with a specific user condition, and the target interactive model. We then extract stepwise model response pairs and construct all extracted chosen-rejected pairs. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** The preference dataset D for DPO alignment comprises two parts: (1) Dp, which consists of pairwise responses extracted via i×MCTS, and (2) Dt, which contains response pairs generated by πθ given contexts from πθ’s training set and ranked by a reward model trained on Dp. We denote the preference dataset used for alignment as D, which consists of two parts, as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Human rating results. identities were kept blind to the RAs. After each conversation, the RA rated the model’s supportiveness and their willingness to interact with it in the future, both on a scale from 1 to 10. The results are shown in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: The prompt for the Persuader LLM. training. Specifically, our target is only to simulate the user’s responses in some specific interactive scenarios (emotional support & persuasion for good) rather than modeling the full range of user behaviors. We evaluated the cons…

**Figure 4.** Figure 4: The prompt for the Support LLM. A.3 Reliability of User Simulator It is a fact that relying solely on LLMs to simulate user behaviors could still bring additional variances. However, our system benefits from the simulator by improving alignment with real user responses…

**Figure 6.** Figure 6: UI designs. to the authors. We collected data from participants who voluntarily chose to take part in the study. A.6 Reproducibility We loaded Qwen-2.5-7B-Instruct (Yang et al., 2024) 1 , Llama-3.2-3B-Instruct (Meta, 2024; Dubey et al., 2024) 2 and all-MiniLM-L6-v2 mod…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 14 canonical work pages

[1]

P Auer. 2002. Finite-time analysis of the multiarmed bandit problem

work page 2002
[2]

Yirong Chen, Xiaofen Xing, Jingkai Lin, Huimin Zheng, Zhenyu Wang, Qi Liu, and Xiangmin Xu. 2023. Soulchat: Improving llms’ empathy, listening, and comfort abilities through fine-tuning with multi-turn empathy conversations. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1170--1183

work page 2023
[3]

Yang Deng, Wenxuan Zhang, Yifei Yuan, and Wai Lam. 2023. Knowledge-enhanced mixed-initiative dialogue system for emotional support conversations. arXiv preprint arXiv:2305.10172

arXiv 2023
[4]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

arXiv 2024
[5]

Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. 2023. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179

arXiv 2023
[6]

Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1):59--109

work page 2004
[7]

Kazuaki Furumai, Roberto Legaspi, Julio Vizcarra, Yudai Yamazaki, Yasutaka Nishimura, Sina J Semnani, Kazushi Ikeda, Weiyan Shi, and Monica S Lam. 2024. Zero-shot persuasive chatbots with llm-generated strategies and information retrieval. arXiv preprint arXiv:2407.03585

arXiv 2024
[8]

Emma Holdsworth, Erica Bowen, Sarah Brown, and Douglas Howat. 2014. Client engagement in psychotherapeutic treatment and associations with client characteristics, therapist characteristics, and treatment factors. Clinical psychology review, 34(5):428--450

work page 2014

Show all 37 references

[9]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

2022
[10]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[11]

Clayton Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216--225

2014
[12]

Zhigen Li, Jianxiang Peng, Yanmeng Wang, Tianhao Shen, Minghui Zhang, Linxi Su, Shang Wu, Yihang Wu, Yuqian Wang, Ye Wang, et al. 2024. Planning with large language models for conversational agents. arXiv preprint arXiv:2407.03884

2024 arXiv
[13]

Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In ACL

2021
[14]

Tianjian Liu, Hongzheng Zhao, Yuheng Liu, Xingbo Wang, and Zhenhui Peng. 2024. Compeer: A generative conversational agent for proactive peer support. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, pages 1--22

2024
[15]

AI Meta. 2024. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. Meta AI Blog. Retrieved December, 20:2024

2024
[16]

Huachuan Qiu and Zhenzhong Lan. 2024. Interactive agents: Simulating counselor-client psychological counseling via role-playing llm-to-llm interactions. arXiv preprint arXiv:2408.15787

2024 arXiv
[17]

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

2024
[18]

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484--489

2016
[19]

Hao Sun, Zhenru Lin, Chujie Zheng, Siyang Liu, and Minlie Huang. 2021. Psyqa: A chinese dataset for generating long counseling text for mental health support. In Findings of the Association for Computational Linguistics: ACL 2021

2021
[20]

Jiashuo Wang, Yi Cheng, and Wenjie Li. 2022. Care: Causality reasoning for empathetic responses by conditional graph generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 729--741

2022
[21]

Jiashuo Wang, Yang Xiao, Yanran Li, Changhe Song, Chunpu Xu, Chenhao Tan, and Wenjie Li. 2024 a . Towards a client-centered assessment of llm therapists by client simulation. arXiv preprint arXiv:2406.12266

2024 arXiv
[22]

Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, and Dong Yu. 2024 b . Towards self-improvement of llms via mcts: Leveraging stepwise knowledge with curriculum preference learning. arXiv preprint arXiv:2410.06508

2024 arXiv
[23]

Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. arXiv preprint arXiv:1906.06725

2019 arXiv
[24]

Haojie Xie, Yirong Chen, Xiaofen Xing, Jingkai Lin, and Xiangmin Xu. 2024 a . http://arxiv.org/abs/2412.13660 Psydt: Using llms to construct the digital twin of psychological counselor with personalized counseling style for psychological counseling

2024 arXiv
[25]

Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. 2024 b . Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2405.00451

2024 arXiv
[26]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

2024 arXiv
[27]

Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. 2021. Mastering atari games with limited data. Advances in neural information processing systems, 34:25476--25488

2021
[28]

Dian Yu, Michelle Cohn, Yi Mang Yang, Chun-Yen Chen, Weiming Wen, Jiaping Zhang, Mingyang Zhou, Kevin Jesse, Austin Chau, Antara Bhowmick, et al. 2019. Gunrock: A social bot for complex and engaging long conversations. EMNLP-IJCNLP 2019, page 79

2019
[29]

Xiao Yu, Maximillian Chen, and Zhou Yu. 2023. Prompt-based monte-carlo tree search for goal-oriented dialogue policy planning. arXiv preprint arXiv:2305.13660

2023 arXiv
[30]

Zhou Yu, Alan W Black, and Alexander I Rudnicky. 2017. Learning conversational systems that interleave task and non-task content. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 4214--4220

2017
[31]

Zhou Yu, Leah Nicolich-Henkin, Alan W Black, and Alexander Rudnicky. 2016. A wizard-of-oz study on a non-task-oriented dialog systems that reacts to user engagement. In Proceedings of the 17th annual meeting of the Special Interest Group on Discourse and Dialogue, pages 55--63

2016
[32]

Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Rest-mcts*: Llm self-training via process reward guided tree search. arXiv preprint arXiv:2406.03816

2024 arXiv
[33]

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

2024 arXiv
[34]

Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 2023. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406

2023 arXiv
[35]

Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. 2024. Hypothesis generation with large language models. arXiv preprint arXiv:2404.04326

2024 arXiv
[36]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...
[37]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

P Auer. 2002. Finite-time analysis of the multiarmed bandit problem

work page 2002

[2] [2]

Yirong Chen, Xiaofen Xing, Jingkai Lin, Huimin Zheng, Zhenyu Wang, Qi Liu, and Xiangmin Xu. 2023. Soulchat: Improving llms’ empathy, listening, and comfort abilities through fine-tuning with multi-turn empathy conversations. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1170--1183

work page 2023

[3] [3]

Yang Deng, Wenxuan Zhang, Yifei Yuan, and Wai Lam. 2023. Knowledge-enhanced mixed-initiative dialogue system for emotional support conversations. arXiv preprint arXiv:2305.10172

arXiv 2023

[4] [4]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

arXiv 2024

[5] [5]

Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. 2023. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179

arXiv 2023

[6] [6]

Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1):59--109

work page 2004

[7] [7]

Kazuaki Furumai, Roberto Legaspi, Julio Vizcarra, Yudai Yamazaki, Yasutaka Nishimura, Sina J Semnani, Kazushi Ikeda, Weiyan Shi, and Monica S Lam. 2024. Zero-shot persuasive chatbots with llm-generated strategies and information retrieval. arXiv preprint arXiv:2407.03585

arXiv 2024

[8] [8]

Emma Holdsworth, Erica Bowen, Sarah Brown, and Douglas Howat. 2014. Client engagement in psychotherapeutic treatment and associations with client characteristics, therapist characteristics, and treatment factors. Clinical psychology review, 34(5):428--450

work page 2014