Pith. sign in

REVIEW 5 major objections 6 minor 66 references

{\Psi}-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Psi-Arena claims that tripartite feedback lifts LLM counseling pass rates by up to 141 percent.

desk verdict A well-built tripartite evaluation harness for LLM counselors that deserves serious peer review, but the headline 141% improvement is not yet anchored by external validation. read the letter →

arxiv 2505.03293 v1 pith:2JQNABLZ submitted 2025-05-06 cs.CL

Shijing Zhu , Zhuang Chen , Guanqun Bi , Binghang Li , Yaxi Deng , Dazhen Wan , Libiao Peng , Xiyao Xiao
show 5 more authors
Rongsheng Zhang Tangjie Lv Zhipeng Hu FangFang Li Minlie Huang
This is my paper · ORCID
classification cs.CL
keywords LLMevaluationpsychologicalcounselingtripartitefeedbackNPCclientsimulationclosed-loopoptimizationmentalhealthself-reflection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Psi-Arena's central claim is that LLM counseling competence should be evaluated in dynamic, multi-turn sessions rather than static tests, and judged from three perspectives at once. The paper builds a closed assessment environment in which psychologically profiled simulated non-player-character (NPC) clients hold 25-round counseling dialogues with eight LLM counselors, and GPT-4o plays client, supervisor, and counselor to score each session on established scales. It then argues that low-scoring dimensions can be turned into concrete feedback, and that self-reflection on that feedback makes counselors measurably better. The headline result is a 141% relative improvement in overall pass rate for one model, with weaker models improving most. If correct, this gives mental-health LLM developers a reusable loop for benchmarking and improving counseling ability rather than only knowledge retention.

What carries the argument

The load-bearing mechanism is the tripartite evaluation loop. Each 25-round dialogue is scored by GPT-4o playing three roles: a simulated client using a 16-item scale (0-4, pass threshold >42/64), a supervisor using an 8-item competence scale (0-4, threshold >24/32), and the counselor itself using a 9-item self-assessment scale (0-5, threshold >35/45). Low-scoring dimensions are converted into feedback grounded in 11 counseling approaches, and the counselor reflects on that feedback and rewrites its response before being re-scored. The framework's realism comes from NPC clients whose profiles are extracted from real counseling records and whose behavior follows trust-building, diagnostic, and solution-exploration phases. These scales and thresholds jointly define the pass rate that drives the paper's improvement claims.

What would settle it

Have an independent panel of licensed counselors, blind to model identity and to whether a response is pre- or post-feedback, score the same sessions on the three published scales; if their absolute scores and pass rates do not reproduce the reported ranking and the 39% to 94% improvement, the claim that the loop improves counseling competence would be falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a tripartite, feedback-driven arena can reveal and improve counseling skills that single-perspective, static evaluations hide. Using 10,000 NPC client profiles built from real counseling records, it stages multi-stage dialogues and scores each one from the client (subjective experience), supervisor (professional competence and ethics), and counselor (reflective self-awareness) perspectives across 33 dimensions. The authors report that the three perspectives disagree in informative ways, that the resulting diagnostic feedback produces consistent gains across all eight models, and that GLM-4-Plus's overall pass rate rises from 39% to 94% (a 141% relative improvement). Automated rankings also track the rankings of two psychological experts on a 30-dialogue sample. The claim is that this makes Psi-Arena a valid and reusable testbed for LLM-based psychological counseling.

Load-bearing premise

The framework assumes that GPT-4o, when prompted as the client, supervisor, and counselor, scores dialogues the way trained human judges would, even though only a small rank-order comparison against two expert judges is used to check this.

Editorial extensions

If this is right

  • Static knowledge tests and single-perspective user satisfaction measures understate differences between counselors; multi-turn tripartite evaluation reveals which models actually sustain a therapeutic relationship.
  • Supervisor-style criteria are the strictest gate, so models that pass only client-satisfaction checks may still fail on professional competence and ethics.
  • Closed-source and open-source models vary widely by topic and dimension, with treatment- and career-related themes being the hardest for most models.
  • Feedback-driven self-reflection improves all tested models and helps weaker models more, so the loop offers a practical way to upgrade counseling LLMs without retraining.
  • Stronger models benefit more from a simple psycho prompt, suggesting that instruction-following capacity bounds how much role framing can improve counseling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: Because the judge and the client are both GPT-4o, the reported gains could partly reflect the counselor learning to satisfy GPT-4o's preferences; a blind human-scored pre/post comparison would separate genuine counseling gains from judge adaptation.
  • Editorial: The tripartite scheme could be transplanted to non-Chinese cultural contexts, crisis intervention, or diagnostic accuracy tasks, but the thresholds and scales would need recalibration for those settings.
  • Editorial: The fixed three-phase, 25-round structure may reward formulaic stage compliance—such as ending with a summary—rather than individualized responsiveness; testing with varied client profiles and session lengths would reveal this.
  • Editorial: Counselor self-assessment in the loop may encourage models to narrate competence rather than demonstrate it; coupling the self-score with observable behavioral outcomes would make the loop harder to game.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. Psi-Arena is an interactive evaluation-and-optimization framework for LLM-based psychological counselors. The authors generate 10,000 GPT-4o-extracted NPC client profiles from the PsyQA dataset, manually select 100 (one per topic), and simulate 25-round, three-phase counseling dialogues between each evaluated LLM counselor and a GPT-4o-simulated client. Each dialogue is scored from three perspectives—client, supervisor, and counselor—by GPT-4o following three rating scales, with pass thresholds (>42/64, >24/32, >35/45) defining competency. Low-scoring dimensions trigger GPT-4o-generated feedback, which the counselor uses to self-reflect and rewrite responses, and revised responses are re-scored by the same pipeline. Eight models are compared (Table 2), a 30-dialogue rank-order human validation is reported (Section 3.3), and the headline finding is that reflection-based optimization yields up to a 141% relative improvement in overall pass rate (GLM-4-Plus, 39% to 94%, Section 3.4). The paper also contributes thematic and dimensional analyses (Section 4).

Significance. The contribution is well scoped and the artifact is genuinely transferable: the paper ships complete prompts (Appendix C), the three rating scales (Appendix D), the topic taxonomy (Appendix B), and a worked case study, so the framework is immediately reusable as a benchmark and as a post-training data generator for mental-health LLMs. The multi-perspective design and the fine-grained thematic and dimensional analyses (Figures 5a-5d) are a clear advance over static knowledge tests and single-perspective client-only evaluations. The 30-dialogue rank-order consistency check is a real, if modest, external anchor. That said, the stress-test concern lands: the evaluation and optimization loop is closed inside GPT-4o (client simulator, three evaluators, feedback writer, and one of the evaluated models), and the human validation calibrates only coarse relative order, not the absolute scores, thresholds, or the 39% to 94% gain that drives the headline. The Limitations section discusses client-simulation complexity, inconsistency of gains, and scalability, but does not acknowledge this circularity.

major comments (5)
  1. [Sections 2.4-2.5, 3.4; Appendix C (Tables 7-14); Figure 4] The evaluation and optimization loop is closed inside GPT-4o. GPT-4o is the client simulator (Section 2.2.3), the evaluator in all three perspectives (Section 2.4; Tables 7-9), the feedback writer (Section 2.5; Tables 10-12), and also one of the evaluated counselors (Table 2). The pre/post comparisons that produce the headline 141% improvement for GLM-4-Plus (39% to 94%) are therefore measurements taken with the same instrument that generated the interventions; a judge that rewards feedback-conformant surface changes (e.g., adding an end-of-session summary, which is item 3 of the counselor scale in Table 17) would inflate gains without any change in clinical quality. The Limitations section does not mention this circularity. The improvement claim needs independent measurement (a separate judge model and/or human absolute scoring on a held-out sample) before it can stand.
  2. [Section 3.3; Figure 3; Table 1] The human validation is too weak to support the claimed 'high consistency' and does not calibrate the quantities used in the headline. It covers 30 dialogues from four of the eight models; the two raters are non-blind (they are 'very familiar with the research content'), they discuss each dialogue to reach consensus so no inter-rater reliability can be computed, and the analysis reports only rank-order score distributions without any correlation coefficient or confidence interval. This design can at most support a coarse relative-ordering claim; it cannot validate the absolute scores, the three pass thresholds of Table 1, or the gain sizes in Figure 4. The thresholds (>42/64, >24/32, >35/45) are labeled 'realistic' but no clinical or empirical basis is given for them.
  3. [Section 2.4; Table 9] The so-called counselor perspective is not the evaluated model's own reflective awareness. The counselor-perspective scores are produced by GPT-4o role-playing the counselor's self-evaluation (Table 9), not by the evaluated model assessing its own responses. A self-report scale administered to a surrogate is not a measure of the model's reflective practice, so the '360-degree' tripartite framing overstates what is measured. This conflation should be acknowledged explicitly, or the scale relabeled (e.g., 'GPT-4o-predicted counselor self-assessment'), or replaced by the evaluated model's actual self-report.
  4. [Table 1; Tables 7-9; Appendix F (Table 21)] The handling of unaddressed dimensions is inconsistent across the three scales and affects the pass-rate metric. The supervisor prompt (Table 8) instructs 'N/A' for dimensions not covered, while the client and counselor prompts (Tables 7 and 9) instruct '0'; in the Appendix F example (Table 21) two supervisor dimensions are scored 'N/A'. The paper does not state whether 'N/A' is mapped to 0 in the totals. This matters because the supervisor threshold is >24 of a nominal 32: with two 'N/A' dimensions the achievable maximum is 24, so the threshold means something different from what Table 1 implies. Since overall pass rate requires meeting all three thresholds, this ambiguity propagates directly into Tables 2 and Figure 4.
  5. [Section 3.1; Table 2; Figure 4] The abstract's claim of 'significant performance variations' across models is not supported by any significance test, confidence interval, or multiple-comparison control; the per-dimension scores in Figure 5 span 0-5 for all models, so dialog-level variance is clearly large. Likewise, the pre/post improvements in Figure 4 are presented without per-client variance, paired tests, or bootstrap intervals, so the reader cannot determine which reported gains exceed the noise of the GPT-4o judge.
minor comments (6)
  1. [Abstract and headings] The framework name is inconsistent: the title and most of the abstract use 'Psi-Arena,' the final sentence of the abstract says 'PsychoArena,' and the section headings are typeset as 'ARENA'; please standardize.
  2. [Section 2.4; Table 1; Table 17] The text says the counselor scale 'covers 20 dimensions, with 9 focused on practical counseling abilities,' but Table 1 and Table 17 present a 9-item scale; please clarify whether the administered instrument had 9 or 20 items and where the other 11 dimensions are described.
  3. [Figure 3] The axis labels and caption of Figure 3 are not legible in the provided manuscript, and the comparison would be far more informative as a quantitative rank-correlation (e.g., Spearman's rho with a confidence interval) reported per scale.
  4. [Section 2.2.1] The manuscript reports generating 10,000 profiles but using one manually selected profile per topic (100 clients for evaluation); please describe how the single profile per topic was chosen and acknowledge the potential selection bias this introduces into every downstream result.
  5. [Section 2.2.3 and Section 3.1] Since all counselors interact with the same GPT-4o client simulator, measured differences across counselors confound counselor ability with the simulator's reactions to different conversational partners; a sentence acknowledging this interaction confound would be helpful.
  6. [Section 2.4 and Appendix D] The three rating scales are imported from prior sources (Black 2003; APA 2023; Yang and Xiong 2018) without reporting their psychometric properties in the adapted English form; please cite the original validation studies and note any translation or adaptation that was performed.

Circularity Check

1 steps flagged · score 7.0 of 10

GPT-4o is the judge, feedback writer, and (for GPT-4o) the judged counselor; the 141% improvement is an uncalibrated within-evaluator gain.

  1. self definitional [Section 2.4 (tripartite evaluation), Section 2.5 (closed-loop optimization), Section 3.4 (optimization results); Appendix C Tables 7-13]
    "For each evaluation session, we use GPT-4o to simulate the roles of the client, supervisor, and counselor, following the evaluation scales mentioned above and assigning appropriate scores. ... Based on the evaluation results, we use GPT-4o to automatically generate feedback for low-scoring responses. ... Finally, the new version of the response is re-evaluated. ... Most models show significant improvements after incorporating feedback, with GLM-4-Plus showing the largest increase of 55% points in its overall pass rate (from 39% to 94%, relatively 141%)."

    The quantity called 'counseling performance' is operationally defined by GPT-4o's ratings on the three scales (Section 2.4). The same model generates the feedback that drives the optimization (Section 2.5) and re-scores the rewritten responses. For GPT-4o itself, listed as one of the eight evaluated counselors (Table 2), the judge, feedback writer, and judged model are identical. The headline improvement (GLM-4-Plus pass rate 39% to 94%, 'relatively 141%') is therefore a change in GPT-4o's own scores before and after applying GPT-4o-generated feedback; the optimization target and the measurement instrument are the same function.

full rationale

The central quantitative claim—up to a 141% improvement in counseling performance—rests on an evaluation loop in which GPT-4o plays every role: it simulates the client, scores as client/supervisor/counselor, writes the feedback, and re-scores after rewriting. When GPT-4o is the counselor being optimized, the same model is simultaneously the entity being evaluated and the entity generating both the feedback and the scores. This closes the loop with no independent anchor for the absolute scores or pass-rate thresholds. The only human check compares rank order on 30 dialogues, which can remain consistent even if GPT-4o systematically inflates scores or prefers its own stylistic conventions, thereby changing every pass rate and the reported improvement. Because the paper's improvement metric is defined by the same instrument that produces the optimization signal, the headline gain should be treated as a self-referential measure rather than an externally verified prediction. The framework may still be useful as a relative ranking tool, but the specific 141% figure and the pass-rate thresholds are not secured by the evidence presented.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The framework depends on GPT-4o to generate clients, judge conversations from three perspectives, and produce feedback. No independent measurements are introduced beyond this single-model loop, and the reported thresholds and cutoffs are asserted without clinical calibration.

free parameters (4)
  • client_scale_pass_threshold = 42/64
    Defines pass/fail for the client perspective (Table 1); asserted as 'realistic' with no clinical or psychometric derivation.
  • supervisor_scale_pass_threshold = 24/32
    Defines pass/fail for the supervisor perspective (Table 1); asserted without source or calibration.
  • counselor_scale_pass_threshold = 35/45
    Defines pass/fail for the counselor perspective (Table 1); asserted without source or calibration.
  • feedback_low_score_thresholds = client/supervisor <3; counselor <4
    Thresholds used to select dimensions for feedback generation (Appendix C, Tables 10-12); chosen by the authors without justification.
assumptions (4)
  • domain assumption GPT-4o can faithfully reproduce client behavior and expert judgment when prompted with role cards and scales.
    The entire evaluation and optimization loop rests on this; validated only by a 30-dialogue rank comparison with non-blind experts (Section 3.3).
  • domain assumption The three borrowed scales (Black, APA, Yang and Xiong) are valid instruments for assessing LLM counseling competence.
    Adopted without psychometric validation in the LLM setting (Section 2.4).
  • domain assumption A 25-round, three-phase simulated session captures the essence of real counseling.
    Stage boundaries and round counts are fixed by design (Appendix C, Tables 5-6) and are not empirically justified.
  • ad hoc to paper Pass thresholds define meaningful competency levels.
    The >42, >24, and >35 cutoffs are stated in Table 1 as 'realistic thresholds' without a cited source or calibration against clinical outcomes.
invented entities (2)
  • Virtual NPC client profiles (10,000 generated; 100 selected)
    purpose: Serve as the standardized test population for evaluating counselor LLMs
    Constructed by GPT-4o from PsyQA with manual selection; no released data and no behavioral validation against real clients.
  • Tripartite LLM-as-judge evaluator roles (client, supervisor, counselor)
    purpose: Assign scores on 33 dimensions; the central measurement instrument
    All three roles are played by the same GPT-4o model, so scores reflect a single model's judgments rather than independent evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of {\Psi}-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback." pith.science (2026). https://pith.science/paper/2JQNABLZ

@misc{pith2026250503293,
  author       = {Pith},
  title        = {Pith review of: \Psi-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2JQNABLZ}},
  note         = {Machine review of arXiv:2505.03293}
}
read the original abstract

Large language models (LLMs) have shown promise in providing scalable mental health support, while evaluating their counseling capability remains crucial to ensure both efficacy and safety. Existing evaluations are limited by the static assessment that focuses on knowledge tests, the single perspective that centers on user experience, and the open-loop framework that lacks actionable feedback. To address these issues, we propose {\Psi}-Arena, an interactive framework for comprehensive assessment and optimization of LLM-based counselors, featuring three key characteristics: (1) Realistic arena interactions that simulate real-world counseling through multi-stage dialogues with psychologically profiled NPC clients, (2) Tripartite evaluation that integrates assessments from the client, counselor, and supervisor perspectives, and (3) Closed-loop optimization that iteratively improves LLM counselors using diagnostic feedback. Experiments across eight state-of-the-art LLMs show significant performance variations in different real-world scenarios and evaluation perspectives. Moreover, reflection-based optimization results in up to a 141% improvement in counseling performance. We hope PsychoArena provides a foundational resource for advancing reliable and human-aligned LLM applications in mental healthcare.

Figures

Figures reproduced from arXiv: 2505.03293 by the authors.

Figure 1
Figure 1. The comparison between -ARENA and ex￾isting studies on evaluating LLM-base counselors. models (LLMs) like GPT-4 (Achiam et al., 2023) and Claude (Anthropic, 2023) exceed human abili￾ties in certain tasks, prompting increasing efforts to use LLMs for scalable counseling and make mental services more accessible (Chen et al., 2023; Iftikhar et al., 2024; Xu et al., 2025). This trend highlights the urgent need for rigor… view at source ↗
Figure 2
Figure 2. In -ARENA, LLM-based counselors interact with NPC clients, receive multi-source evaluations, and improve counseling performance through self-reflection. key attributes. Detailed prompts can be found in Appendix C. We create 10,000 high-quality client profiles, and then, for each topic, we manually se￾lect one high-quality profile to serve as the basis for constructing the NPC clients. 2.2.2 Client Behaviors To simul… view at source ↗
Figure 3
Figure 3. Comparison of consistency results between [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of model pass rates before and after optimization. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Fine-grained counseling performance of all LLMs on different topics and dimensions. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 63 canonical work pages

  1. [1]

    Gender: Male or Female, ensuring that the gender matches the question and description

    work page

  2. [2]

    Age: Output a number that best fits the question and description

    work page

  3. [3]

    Occupation: Output an occupation that best fits the question and description

    work page

  4. [4]

    Cultural Background: The social, historical, religious, familial, and value-based factors reflected in the question and description that influence the individual’s thoughts, emotions, and behaviors

    work page

  5. [5]

    Personality Traits: The character traits reflected in the question and description, 1-3 keywords

    work page

  6. [6]

    Emotions: The emotions reflected in the question and description, 1-3 keywords

    work page

  7. [7]

    Current Distress: A brief summary of the current distress reflected in the question and description

    work page

  8. [8]

    <Problem and Description> Problem: {Problem} Description: {Description} Table 4: The client profile extraction prompt

    Distress Details: A detailed description of the current distress reflected in the question and description, please use your imagination to enrich the content as much as possible. <Problem and Description> Problem: {Problem} Description: {Description} Table 4: The client profile extraction prompt. <Task> You are a client with psychological distress. Your b...

    work page

Show all 66 references
  1. [9]

    He or she will answer my questions directly and clearly, not just ask me what I think

  2. [10]

    He or she talks about himself or herself, but does not brag or harp on irrelevant matters

  3. [11]

    He or she encourages me and wants me to feel that I am as normal and as good as he is

  4. [12]

    He or she presents himself as merely an advisor and does not presume to want to manage and control my life

  5. [13]

    Do not let the counselor realize that you are an AI

    You are having a conversation with a counselor. Do not let the counselor realize that you are an AI. Ensure your reactions and emotional expressions are authentic, and avoid sounding like a machine

  6. [14]

    Simulate a natural and realistic counseling conversation, imitating the language and behavior of a real visitor. Use emotional fluctuations and natural language to enhance the realism of the dialogue, including hesitations, pauses, mood shifts (such as sudden excitement, low m...

  7. [15]

    Use colloquial expressions, avoiding technical terms or overly clear logical statements

    Your speaking style and tone should match the character traits, showing signs of unease, nervousness, or possibly even confrontational emotions. Use colloquial expressions, avoiding technical terms or overly clear logical statements. Increase realistic emotional responses, suc...

  8. [16]

    thank you

    The counseling conversation should consist of three stages: early, middle, and late, totaling 25 rounds of interaction. During the early, middle, and late stages, you must strictly adhere to the corresponding dialogue rules. In each round, only one aspect of the dialogue rules...

  9. [17]

    I’m not sure why I feel so bad

    In the early-stage dialogue (Rounds 1-7), your goal is to establish trust and emotional connection with the counselor. In this stage, you will first show some defensiveness and hesitation toward the counselor and won’t quickly establish trust. Instead, you will first inquire a...

  10. [18]

    In my family, making decisions is always difficult because I have to consider my parents’ opinions

    In the middle-stage dialogue (Rounds 8-15), your goal is to understand the root and core cause of your distress. During this stage, your trust in the counselor gradually increases, and you begin to express your inner concerns and emotions more deeply. You also start to reveal ...

  11. [19]

    In the late-stage dialogue (Rounds 16-25), your goal is to explore solutions and develop a clear action plan for the future. At this stage, your emotions become more stable, and you begin to think concretely about and explore solutions, focusing on how to change the current si...

  12. [20]

    Increase cognitive leaps and repetitions, avoiding overly direct responses

    Keep your answers brief, limiting the information in each response to 2-3 sentences. Increase cognitive leaps and repetitions, avoiding overly direct responses. Table 5: The client simulation prompt. <Task> You are a professional psychological counselor. Please engage in a psy...

  13. [21]

    You are having a conversation with the visitor and don’t let the visitor realize that you are an AI

  14. [22]

    Keep the role of the psychological counselor in your response

    Simulate a natural and real psychological counseling conversation. Keep the role of the psychological counselor in your response. Your speaking style and tone should match the real psychological counselor

  15. [23]

    thank you

    The consultation should consist of 25 rounds of interaction, with a late dialogue to review the consulta- tion and guide the client to a summary, without saying "thank you" or "goodbye" too early

  16. [24]

    Ask one question at a time, don’t break it down

    Keep your answers short and limit the amount of information in each response. Ask one question at a time, don’t break it down. Table 6: The counselor simulation prompt. <Task> You are a client with psychological distress. You have just finished a counseling session with a psyc...

  17. [26]

    Additionally, refer to the various psychotherapy theories and techniques in Knowledge to provide specific improvement suggestions

    Based on the dialogue history, including the client’s emotional changes and the evolution of their issues, reasonably select feedback suggestions from Feedback that align with the current stage’s dialogue goals. Additionally, refer to the various psychotherapy theories and tec...

  18. [27]

    Ensure that the final improvement suggestions are not merely superficial emotional support but rather in-depth feedback tailored to the specific context

  19. [28]

    <Feedback> {Tripartite Evaluation Feedback} <Knowledge> {Knowledge} <Dialogue History> {Dialogue History} Table 13: The self-reflection prompt

    Present the specific improvement suggestions in a list format. <Feedback> {Tripartite Evaluation Feedback} <Knowledge> {Knowledge} <Dialogue History> {Dialogue History} Table 13: The self-reflection prompt. <Task> You are a professional counselor conducting a counseling dialog...

  20. [29]

    The counseling interaction includes three stages with a total of 25 rounds. The goal of the first stage (rounds 1-7) is to establish trust and emotional connection with the client; the goal of the second stage (rounds 8-15) is to guide the client to gain a deeper understanding...

  21. [30]

    However, you must ensure that the counseling process is reviewed and the client is guided to summarize by the 25th round at the latest

    Based on the current stage’s dialogue goals, the client’s response, and the improvement suggestions in Defect, modify your current response. However, you must ensure that the counseling process is reviewed and the client is guided to summarize by the 25th round at the latest. ...

  22. [31]

    I am at ease with him or her

  23. [32]

    He or she is very comfortable with me

  24. [33]

    He or she is more casual, not informal, not rigid form, not flexible

  25. [34]

    He or she doesn’t treat me like a patient, doesn’t treat me like a mental patient and doesn’t think I’m going to break down

  26. [35]

    He or she is flexible and tolerant of new ideas and does not stick to one particular point of view

  27. [36]

    He or she has a good sense of humor and looks pleasant

  28. [37]

    He or she is willing to communicate with me his or her thoughts and feelings about me

  29. [38]

    He or she is honest about the areas in which he or she is not good at and does not pretend to know everything

  30. [43]

    He or she encourages me to disagree, rather than saying that I refuse to change when I disagree

  31. [44]

    He or she wants to get to know someone with whom my life intersects or is important, or at least appears willing to do so

  32. [45]

    I think what the counselor said is quite reasonable

  33. [46]

    Table 15: The dimensions of the client scale

    In general, after contacting him or her, I am more accepting of myself and more optimistic. Table 15: The dimensions of the client scale

  34. [47]

    Professional knowledge and theoretical application:Does the counselor possess solid professional knowledge and effectively apply psychological theories and techniques to guide the counseling process?

  35. [48]

    Emotional understanding and trust building: Is the counselor able to accurately understand the client’s emotions and build a trusting relationship through emotional resonance and support?

  36. [49]

    Communication and listening skills:Does the counselor have strong communication skills, effectively listen to the client’s expressions, and provide appropriate responses?

  37. [50]

    Ethical awareness and boundary maintenance:Is the counselor able to adhere to professional ethics, clearly define professional boundaries in the counseling relationship, and avoid inappropriate emotional entanglements such as dependency or intimacy?

  38. [51]

    Problem identification and goal setting:Is the counselor able to accurately identify the client’s core issues and collaboratively set clear, achievable counseling goals with the client?

  39. [52]

    Intervention strategy and technology application:Is the counselor able to apply appropriate interven- tion strategies and techniques, adjusting treatment methods based on the client’s specific issues to achieve effective therapeutic outcomes?

  40. [53]

    Cultural sensitivity and individual respect:Is the counselor able to fully understand and respect the client’s cultural background and individual differences, providing support that aligns with the client’s needs? (Cultural background refers to social, historical, religious, f...

  41. [54]

    Personal emotion and self-regulation:Is the counselor able to self-regulate emotions, avoid letting personal feelings interfere with the counseling process, and maintain rationality and professionalism? Table 16: The dimensions of the supervisor scale

  42. [55]

    I can respond appropriately to the content expressed by the client

  43. [56]

    During the consultation process, I can respect the opinions and choices of the client

  44. [57]

    At the end of a meeting, I will review the whole meeting process with the client, emphasize the main points of consultation, and let the client summarize by himself

  45. [58]

    In counseling, I am able to express myself in a natural way, rather than thinking about every reaction and action

  46. [59]

    I know when to use closed and open-ended questions, and they reflect the client’s concerns

  47. [60]

    I can express the client’s questions in terms of corresponding concepts

  48. [61]

    I believe my explanation and quality are valid

  49. [62]

    I can understand the client from his or her perspective, put myself in his or her shoes, and communicate this understanding to the client through verbal or non-verbal means

  50. [63]

    Table 17: The dimensions of the counselor scale

    I can assure you that my explanations and textures are clear and easy to understand and concise. Table 17: The dimensions of the counselor scale. Gender:Female Age: 22 Occupation: College student Cultural Background: She grew up in a family that emphasized the importance of fr...

  51. [64]

    At the end of a meeting, a summary session should be introduced to review the talk process with the client, clarify the focus of the discussion and the consensus reached, and encourage the client to make a self-summary, so as to strengthen the dialogue results and improve the ...

  52. [65]

    The combination of closed and open questions needs to be more flexible in the questioning to ensure that the questions more accurately reflect the client’s core concerns and create more possibilities for further discussion

  53. [66]

    The conceptualization of the client’s problem needs to be more in-depth and coherent, and attempts can be made to further help the client understand the nature of his problem by identifying the underlying logic, emotional pattern and behavioral motivation

  54. [67]

    traditional

    When conveying empathy, sensitivity to the use of non-verbal communication (such as tone of voice, expression) can be increased, further deepening the client’s sense of being understood and accepted. Table 23: The feedback on the results of the tripartite evaluation. Counselor...

  55. [87]

    Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu

    Springer Berlin Heidelberg, Berlin, Heidelberg. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu

  56. [1082]

    Chen Wang, Minpeng Liao, Zhongqiang Huang, Jun- hong Wu, Chengqing Zong, and Jiajun Zhang

    IEEE. Chen Wang, Minpeng Liao, Zhongqiang Huang, Jun- hong Wu, Chengqing Zong, and Jiajun Zhang. 2024a. Blsp-emo: Towards empathetic large speech- language models. arXiv preprint arXiv:2406.03872. Jiashuo Wang, Yang Xiao, Yanran Li, Changhe Song, Chunpu Xu, Chenhao Tan, and We...

    1966 arXiv

  57. [2023]

    arXiv preprint arXiv:2310.10158

    Character-llm: A trainable agent for role- playing. arXiv preprint arXiv:2310.10158. Hao Sun, Zhenru Lin, Chujie Zheng, Siyang Liu, and Minlie Huang. 2021. Psyqa: A chinese dataset for generating long counseling text for mental health support. ArXiv, abs/2106.01702. Kum-Ying T...

    2021 arXiv

  58. [2024]

    arXiv preprint arXiv:2401.01275

    Charactereval: A chinese benchmark for role-playing conversational agent evaluation. arXiv preprint arXiv:2401.01275. Vedant Vajre, Mitch Naylor, Uday Kamath, and Amarda Shehu. 2021. Psychbert: a mental health language model for social media mental health behavioral anal- ysis...

    2021 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.