Pith. sign in

REVIEW 4 major objections 6 minor 56 references

From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning large language models on raw r/Anxiety posts improves readability but raises toxicity and bias and lowers empathy, according to this evaluation.

desk verdict A genuinely useful empirical warning about fine-tuning on raw Reddit data, but the empathy-regression numbers need fixing before the headline claim is credible. read the letter →

arxiv 2505.18464 v1 pith:7QEEGLNV submitted 2025-05-24 cs.HC cs.AIcs.CLcs.CY

Ugur Kursuncu , Trilok Padhi , Gaurav Sinha , Abdulkadir Erol , Jaya Krishna Mandivarapu , Christopher R. Larrison This is my paper
classification cs.HCcs.AIcs.CLcs.CY
keywords largelanguagemodelsmentalhealthanxietysupportfine-tuningsocialmediadatatoxicityempathyhumanevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether fine-tuning large language models on naturalistic social-media discourse—real posts and comments from the r/Anxiety subreddit—makes them better or worse at providing anxiety support. It evaluates GPT-3.5 and Llama 2, each in base and fine-tuned form, on linguistic quality, safety and trustworthiness, and supportiveness, using automated metrics plus ratings from mental-health professionals. The central finding is that fine-tuning improves readability and fluency, especially for GPT, but increases toxicity and bias and reduces emotional responsiveness, particularly reflection and sympathy. A sympathetic reader should care because this is evidence that raw community data is not a safe training target for mental-health chatbots without explicit mitigation.

What carries the argument

The argument is carried by a paired before/after comparison held fixed across prompts: each model is evaluated in base and fine-tuned form on the same set of r/Anxiety posts, so any change is attributable to the fine-tuning data. Fine-tuning minimizes a standard language-model loss on prompt-response pairs where the responses are Reddit comments with more than four net upvotes, the paper's proxy for quality. The evaluation itself is a three-axis rubric—readability, coherence, and semantic consistency; toxicity and bias; and empathy, reflection, and sympathy—with automated scores supplemented by clinician ratings. This setup isolates the effect of adding raw social-media text to a general-purpose model.

What would settle it

Fine-tune the same two base models on the same posts after the comments have been filtered by licensed clinicians for supportiveness and safety, then rerun the toxicity, bias, and empathy measures; if the filtered models still show higher toxicity and lower empathy than the base models, the paper's attribution to unprocessed social-media content would be wrong, and the harm would instead come from fine-tuning itself.

Watch

Extended reading notes

Core claim

Put directly: taking an LLM and fine-tuning it on upvoted r/Anxiety exchanges makes its replies easier to read but more likely to contain toxic or biased language, and measurably less empathetic. GPT-3.5 fine-tuned shows the largest readability gains and also the largest toxicity increases (insult +200%, profanity +350%), while fine-tuned Llama 2 remains less toxic but still degraded; both fine-tuned models score poorly on sympathy in human evaluation. The paper interprets this as evidence that the models absorb stylistic fluency from social-media text at the cost of therapeutic qualities, and that safety guardrails are partially unlearned during fine-tuning. The conclusion is that unprocessed social media data should not be used for domain-specific fine-tuning in mental-health applications without data curation and safety mitigation.

Load-bearing premise

The whole pipeline assumes that Reddit comments with more than four net upvotes are high-quality, supportive responses to anxiety posts, because those same comments are both what the models learn from and the reference answers used to score how supportive the outputs are.

Editorial extensions

If this is right

  • Developers who fine-tune mental-health chatbots on unfiltered social media should expect a safety regression, not just a fluency gain, because the paper measures both in the same models.
  • A base model may already match or beat a fine-tuned one on supportiveness: human raters judged non-fine-tuned Llama 2 most empathetic and non-fine-tuned GPT most reflective, while both fine-tuned models lagged on sympathy.
  • The largest supportiveness losses are in the highest-level behaviors: reflection scores fell by about 71% for GPT and 38% for Llama 2 after fine-tuning.
  • The same three-part rubric can be applied to newer or larger models to check whether scaling changes the fine-tuning trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Reddit upvotes do not actually track therapeutic quality, then the paper's 'quality' filter is doing double duty as training signal and gold reference, which would weaken both the fine-tuning and the evaluation; a natural extension is to repeat the study with clinician-curated responses and compare results.
  • The same fine-tuning trade-off may appear in other emotionally sensitive domains such as grief, addiction, or crisis support, where readability gains are less valuable than safety; this is an extrapolation from the anxiety case, not a result the paper demonstrates.
  • The low inter-rater agreement on 'practical and relevant responses' (49.7%) suggests the construct of response quality is not yet well-defined enough for automated benchmarking; a useful next step would be to anchor the rubric in observable user outcomes, such as whether a support-seeker continues the conversation or reports feeling heard.
  • A testable extension: measure whether fine-tuned models' higher toxicity is concentrated in particular prompt types, such as crisis disclosures or identity disclosures, which would indicate where mitigation filters should be inserted.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper evaluates GPT-3.5 and Llama 2, before and after fine-tuning on r/Anxiety prompt-response pairs, using a multidimensional battery of automatic metrics (readability indices, semantic coherence, BERTScore/BLEURT/ROUGE, Perspective API toxicity, GenBit bias, and in-house EPITOME/PAIR supportiveness scores) together with a qualitative review by clinical social workers. It reports that fine-tuning improves readability and fluency, increases toxicity and bias, and diminishes empathy, reflection, and exploration, concluding that fine-tuning on unprocessed social-media data for mental-health support carries safety risks. The main contributions are the naturalistic dataset, the three-category evaluation framework, and the comparison of fine-tuned versus base models from two LLM families.

Significance. If the central claims hold, the paper gives useful, ecologically valid evidence about the risks of domain-specific fine-tuning for mental-health chatbots, and the evaluation framework is a reusable structure for future work. The toxicity/bias leg of the argument is built on standard, reproducible tools (Perspective API, GenBit), and the inclusion of clinician qualitative evaluation is a strength. However, the emotional-responsiveness claim currently rests on an in-house scorer with no reported validation and on internally inconsistent reported values; until those are resolved, the paper's headline conclusion is not fully established. The framework and dataset make this a worthwhile empirical contribution that can be strengthened with additional validation and reporting.

major comments (4)
  1. [§3.5 and Appendix D] The statistical analysis section promises Welch's ANOVA, Games-Howell post-hoc tests, and Hedges' g effect sizes, but none of these outputs appear in the main text or appendix. Table 2 reports only ranks and ranking scores, and Table D2 reports raw means and percentage changes. Without p-values, confidence intervals, and effect sizes, claims such as 'significantly improved' in §4.1.1 and 'ensuring statistically significant comparisons' in the Table 2 note are unsupported. Please report the full statistical results in the appendix, or explicitly relabel the findings as descriptive comparisons.
  2. [§4.3.1, Table D2, Table 2] The Exploration results are internally inconsistent. Section 4.3.1 states that NFT-GPT achieved Exploration = 0.26, that fine-tuning decreased GPT by -15.38%, and that it improved Llama 2 by +45.45%. Table D2 gives Exploration values of 0.07 (FT-GPT), 0.13 (FT-Llama), 0.24 (NFT-GPT), and 0.21 (NFT-Llama), which imply changes of -70.83% and -38.10% for GPT and Llama, respectively. Table 2's Exploration ranks (FT-GPT = 2, FT-Llama = 3, NFT-GPT = 1, NFT-Llama = 4) also contradict a higher-is-better ranking of these raw values. Because 'diminished emotional responsiveness' is a central conclusion, the correct values, calculations, and ranks must be reconciled, or the claim must be restricted to the internally consistent dimensions.
  3. [§3.3.3 and Appendix C] The quantitative supportiveness scores rest on two models whose validity in this setting is not established. The EPITOME-based empathy scorer is a fine-tuned Llama 3.1 model with no reported validation against human expert ratings, and the PAIR reflection model is applied outside its motivational-interviewing training domain without evidence of transfer validity. These scores are the basis for the 'diminished emotional responsiveness' claim in the abstract and Section 5. Please provide a validation study, for example by correlating the automated scores with the clinician ratings from Section 3.4 on a held-out set, or present the supportiveness findings as exploratory and soften the corresponding conclusions.
  4. [§3.1] The dataset construction assumes that a Reddit comment score greater than four identifies 'quality interactions,' and the retained comments serve both as fine-tuning targets and as the reference answers for semantic consistency scoring. No evidence is given that community upvotes track therapeutic quality. If upvotes do not align with supportive or clinically appropriate responses, the training signal, the gold references for BERTScore/BLEURT/ROUGE, and the supportiveness evaluation all inherit that bias. Please add a validation of this assumption (e.g., a small clinician-rated sample) or explicitly discuss how this limitation affects the interpretation of the fine-tuning effects.
minor comments (6)
  1. [Table 2 note] The note says 'refer to Appendix D2' but the table is labeled Table D2; please fix the cross-reference.
  2. [§4.1.2] The text says fine-tuned versions 'produced higher coherence scores' while immediately reporting large negative Cnpmi changes (e.g., -237.5% for GPT); please clarify whether the sentence refers only to Cv or correct the wording so the results are not contradictory.
  3. [§4.2.2] The sentence 'While non-fine-tuned GPT was less toxic than fine-tuned Llama 2 across most metrics, fine-tuned Llama 2 consistently performed better' is contradictory as written; please clarify which comparison is intended.
  4. [§3.2] Fine-tuning hyperparameters are not reported (e.g., LoRA rank and alpha, learning rate, number of epochs, batch size). Including these would materially improve reproducibility.
  5. [§3.4 and Table 3] Inter-rater reliability is reported as percent agreement only. For dimensions such as 'practical/relevant' with 49.7% agreement, chance-corrected coefficients (e.g., Cohen's kappa) should be reported, and the handling of the low-agreement dimension should be described.
  6. [Abstract and §5] The abstract states that fine-tuning 'diminished emotional responsiveness' and Section 5 says it 'diminishes the capacity for empathy and sympathy substantially,' but Table 3 shows fine-tuned GPT at 70.4% empathetic and fine-tuned Llama 2 at 89.5% empathetic while sympathy is low for all models; please align the wording with the full pattern of quantitative and qualitative results.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the main findings rest on held-out data and external scorers; supportiveness-metric inconsistencies and the unvalidated empathy scorer are correctness concerns, not circularity.

full rationale

This paper is an empirical evaluation, not a derivation. The central claims—fine-tuning improves readability and fluency, increases toxicity and bias, and diminishes emotional responsiveness—are produced by applying external, pre-existing metrics (Perspective API, GenBit, readability indices, BERTScore/BLEURT/ROUGE, the EPITOME-based empathy scorer, and the PAIR reflection scorer) to responses generated from a held-out test split. The fine-tuning objective is standard negative log-likelihood on 21,000 Reddit prompt–response pairs, and the 5,000 test instances are disjoint from the training set, so the linguistic-quality improvements are not forced by construction. The empathy and reflection scorers were trained on public empathy corpora (EPITOME and PAIR), not on the GPT/Llama outputs being scored, so the supportiveness findings do not reduce to fitting the data. Self-citations (e.g., Kursuncu et al. 2021; Sinha et al. 2023; Saha et al. 2025; Khandelwal et al. 2024) appear as background about naturalistic social-media language or as recommendations for future mitigation techniques; none supplies the load-bearing evidence for the paper's findings. There are genuine validity concerns that should be weighed elsewhere: the in-house Llama 3.1 empathy scorer has no reported validation against expert clinician ratings, PAIR is applied outside its motivational-interviewing training domain, and the Exploration values are internally inconsistent (§4.3.1 reports -15.38% for GPT and +45.45% for Llama 2, while Table D2 values 0.07/0.24 and 0.13/0.21 imply -70.83% and -38.10%, respectively, and Table D2's Exploration row is identical to Reflection). These are correctness and reproducibility problems in the supportiveness pillar, not circularity: the reported numbers are not equivalent to the inputs by definition. No load-bearing step reduces to a fitted parameter renamed as a prediction, a self-citation chain, or a definitional identity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper's central claim depends on treating upvoted Reddit comments as gold-standard supportive text and on treating automated and in-house metrics as valid measurements of quality, safety, and empathy. No model parameters are fitted in the paper; the comment-score threshold is a hand-chosen filter that shapes the dataset and therefore the results.

free parameters (1)
  • Reddit comment score threshold = 4 (upvotes minus downvotes)
    Hand-chosen in Section 3.1 to define 'quality interactions.' It determines which posts and comments enter the fine-tuning set and the reference set, so it indirectly shapes all evaluation results. No sensitivity analysis is reported.
assumptions (4)
  • domain assumption Comments with score >4 on r/Anxiety are high-quality supportive responses suitable as fine-tuning targets and reference gold labels.
    Section 3.1 uses this threshold to curate 26,450 posts; these comments define both the training signal and the reference responses for semantic consistency metrics.
  • domain assumption The selected automated metrics (readability, coherence, BERTScore, BLEURT, ROUGE, Perspective API, GenBit, EPITOME, PAIR) are valid proxies for anxiety support quality, safety, and supportiveness.
    Sections 3.3.1 to 3.3.3 assume these metrics capture meaningful properties of mental-health support text without validating them against user outcomes or therapeutic effectiveness.
  • ad hoc to paper The fine-tuned Llama 3.1 empathy scorer provides reliable empathy scores without external validation.
    Section 3.3.3 introduces a multi-task fine-tuned Llama 3.1 model trained on public corpora plus in-house annotations. No accuracy, inter-rater agreement, or validation against clinician judgments is reported.
  • domain assumption Psychotherapy common factors and the STAR scale, designed for human therapeutic relationships, transfer to one-shot AI peer-support text.
    The supportiveness framework is grounded in Rogers, Wampold, and STAR, but those instruments were developed for human clinicians; the paper assumes they apply to anonymous AI chat responses.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data." pith.science (2026). https://pith.science/paper/7QEEGLNV

@misc{pith2026250518464,
  author       = {Pith},
  title        = {Pith review of: From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QEEGLNV}},
  note         = {Machine review of arXiv:2505.18464}
}
read the original abstract

The growing demand for accessible mental health support, compounded by workforce shortages and logistical barriers, has led to increased interest in utilizing Large Language Models (LLMs) for scalable and real-time assistance. However, their use in sensitive domains such as anxiety support remains underexamined. This study presents a systematic evaluation of LLMs (GPT and Llama) for their potential utility in anxiety support by using real user-generated posts from the r/Anxiety subreddit for both prompting and fine-tuning. Our approach utilizes a mixed-method evaluation framework incorporating three main categories of criteria: (i) linguistic quality, (ii) safety and trustworthiness, and (iii) supportiveness. Results show that fine-tuning LLMs with naturalistic anxiety-related data enhanced linguistic quality but increased toxicity and bias, and diminished emotional responsiveness. While LLMs exhibited limited empathy, GPT was evaluated as more supportive overall. Our findings highlight the risks of fine-tuning LLMs on unprocessed social media content without mitigation strategies.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 38 canonical work pages

  1. [1]

    APACrefauthors \ 1980

    edition1980diagnostic APACrefauthors APA, A.P.A. APACrefauthors \ 1980 . Diagnostic and statistical manual of mental disorders Diagnostic and statistical manual of mental disorders . American psychiatric association, Washington, DC 205--224,

    work page 1980

  2. [2]

    , Mann, B

    brown2020language APACrefauthors Brown, T. , Mann, B. , Ryder, N. , Subbiah, M. , Kaplan, J.D. , Dhariwal, P. others APACrefauthors \ 2020 . Language models are few-shot learners Language models are few-shot learners . Advances in neural information processing systems 33 1877--1901,

    work page 2020

  3. [3]

    , Wang, X

    chang2023survey APACrefauthors Chang, Y. , Wang, X. , Wang, J. , Wu, Y. , Yang, L. , Zhu, K. others APACrefauthors \ 2023 . A Survey on Evaluation of Large Language Models A survey on evaluation of large language models . arXiv preprint arXiv:2307.03109 ,

    arXiv 2023

  4. [4]

    \ Tai, K.W

    cheung2023use APACrefauthors Cheung, K.K.C. \ Tai, K.W. APACrefauthors \ 2023 . The use of intercoder reliability in qualitative interview data analysis in science education The use of intercoder reliability in qualitative interview data analysis in science education . Research in Science & Technological Education 41 3 1155--1175,

    work page 2023

  5. [5]

    , Dias, L

    Crasto2021CareBotAM APACrefauthors Crasto, R. , Dias, L. , Miranda, D. Kayande, D. APACrefauthors \ 2021 . CareBot: A Mental Health ChatBot Carebot: A mental health chatbot . 2021 2nd International Conference for Emerging Technology (INCET) 1-5, APACrefURL https://api.semanticscholar.org/CorpusID:235617702 APACrefURL

    work page 2021

  6. [6]

    , Levkovich, I

    Elyoseph2024AssessingPI APACrefauthors Elyoseph, Z. , Levkovich, I. Shinan-Altman, S. APACrefauthors \ 2024 . Assessing prognosis in depression: comparing perspectives of AI models, mental health professionals and the general public Assessing prognosis in depression: comparing perspectives of ai models, mental health professionals and the general public ....

    work page 2024

  7. [7]

    , Padhi, T

    Erol2025PlayingDA APACrefauthors Erol, A. , Padhi, T. , Saha, A. , Kursuncu, U. Aktas, M.E. APACrefauthors \ 2025 . Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models Playing devil's advocate: Unmasking toxicity and vulnerabilities in large vision-language models . ArXiv abs/2501.09039 , APACrefURL https://api...

    arXiv 2025

  8. [8]

    \ Howell, J.F

    games1976pairwise APACrefauthors Games, P.A. \ Howell, J.F. APACrefauthors \ 1976 . Pairwise multiple comparison procedures with unequal n’s and/or variances: a Monte Carlo study Pairwise multiple comparison procedures with unequal n’s and/or variances: a monte carlo study . Journal of Educational Statistics 1 2 113--125,

    work page 1976

Show all 56 references
  1. [9]

    , Padhi, T

    garg2024just APACrefauthors Garg, R. , Padhi, T. , Jain, H. , Kursuncu, U. Kumaraguru, P. APACrefauthors \ 2024 . Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes Just kiddin: Knowledge infusion and distillation for detection of indecent memes ....

    2024

  2. [10]

    Let Me Tell You About Your Mental Health!

    Gaur2018LetMT APACrefauthors Gaur, M. , Kursuncu, U. , Alambo, A. , Sheth, A. , Daniulaityte, R. , Thirunarayan, K. Pathak, J. APACrefauthors \ 2018 . "Let Me Tell You About Your Mental Health!": Contextualized Classification of Reddit Posts to DSM-5 for Web-based Intervention...

    2018

  3. [11]

    Perspective API

    perspective_api APACrefauthors Google APACrefauthors \ 2024 . Perspective API. Perspective api. https://perspectiveapi.com/ . Accessed: 2024-09-03

    2024

  4. [12]

    APACrefauthors \ 2023

    Grabb2023TheIO APACrefauthors Grabb, D. APACrefauthors \ 2023 . The impact of prompt engineering in large language model performance: a psychiatric example The impact of prompt engineering in large language model performance: a psychiatric example . Journal of Medical Artifici...

    2023

  5. [13]

    , Jin, R

    guo2023evaluation APACrefauthors Guo, Z. , Jin, R. , Liu, C. , Huang, Y. , Shi, D. , Supryadi others APACrefauthors \ 2023 . Evaluating Large Language Models: A Comprehensive Survey Evaluating large language models: A comprehensive survey . arXiv preprint arXiv:2310.19736 ,

    2023 arXiv

  6. [14]

    , Lai, A

    Guo2024LargeLM APACrefauthors Guo, Z. , Lai, A. , Thygesen, J.H. , Farrington, J. , Keen, T. Li, K. APACrefauthors \ 2024 . Large Language Models for Mental Health Applications: Systematic Review Large language models for mental health applications: Systematic review . JMIR Me...

    2024

  7. [15]

    , Shen, Y

    hu2022lora APACrefauthors Hu, E.J. , Shen, Y. , Wallis, P. , Allen-Zhu, Z. , Li, Y. , Wang, S. others APACrefauthors \ 2022 . Lora: Low-rank adaptation of large language models. Lora: Low-rank adaptation of large language models. ICLR 1 2 3,

    2022

  8. [16]

    , Liu, F

    Hua2024LargeLM APACrefauthors Hua, Y. , Liu, F. , Yang, K. , Li, Z. , Na, H. , han Sheu, Y. Beam, A. APACrefauthors \ 2024 . Large Language Models in Mental Health Care: a Scoping Review Large language models in mental health care: a scoping review . ArXiv abs/2401.02984 , APA...

    2024 arXiv

  9. [17]

    , Bommasani, R

    kapoor2024societal APACrefauthors Kapoor, S. , Bommasani, R. , Klyman, K. , Longpre, S. , Ramaswami, A. , Cihon, P. others APACrefauthors \ 2024 . On the societal impact of open foundation models On the societal impact of open foundation models . arXiv preprint arXiv:2403.07918 ,

    2024 arXiv

  10. [18]

    , Gaur, M

    khandelwal2024domain APACrefauthors Khandelwal, V. , Gaur, M. , Kursuncu, U. , Shalin, V.L. Sheth, A.P. APACrefauthors \ 2024 . A domain-agnostic neurosymbolic approach for big social data analysis: Evaluating mental health sentiment on social media during covid-19 A domain-ag...

    2024

  11. [19]

    , Purohit, H

    kursuncu2021bad APACrefauthors Kursuncu, U. , Purohit, H. , Agarwal, N. Sheth, A. APACrefauthors \ 2021 . When the bad is good and the good is bad: understanding cyber social health through online behavioral change When the bad is good and the good is bad: understanding cyber ...

    2021

  12. [20]

    APACrefauthors \ 2004

    lin2004rouge APACrefauthors Lin, C Y. APACrefauthors \ 2004 . Rouge: A package for automatic evaluation of summaries Rouge: A package for automatic evaluation of summaries . Text summarization branches out Text summarization branches out \ ( \ 74--81)

    2004

  13. [21]

    LLMEval: A Preliminary Study on How to Evaluate Large Language Models Llmeval: A preliminary study on how to evaluate large language models

    llmeval2024guidelines APACrefauthors LLMEval APACrefauthors \ 2024 . LLMEval: A Preliminary Study on How to Evaluate Large Language Models Llmeval: A preliminary study on how to evaluate large language models . arXiv preprint arXiv:2312.07398 ,

    2024 arXiv

  14. [22]

    , Aribandi, V

    Manas2020KnowledgeInfusedAS APACrefauthors Manas, G. , Aribandi, V. , Kursuncu, U. , Alambo, A. , Shalin, V.L. , Thirunarayan, K. Sheth, A. APACrefauthors \ 2020 . Knowledge-Infused Abstractive Summarization of Clinical Diagnostic Interviews: Framework Development Study Knowle...

    2020

  15. [23]

    , Coverdale, J.H

    mccullough2020trustworthiness APACrefauthors McCullough, L.B. , Coverdale, J.H. Chervenak, F.A. APACrefauthors \ 2020 . Trustworthiness and professionalism in academic medicine Trustworthiness and professionalism in academic medicine . Academic Medicine 95 6 828--832,

    2020

  16. [24]

    , McCABE, R

    mcguire2007new APACrefauthors McGuire-Snieckus, R. , McCABE, R. , Catty, J. , Hansson, L. Priebe, S. APACrefauthors \ 2007 . A new scale to assess the therapeutic relationship in community mental health care: STAR A new scale to assess the therapeutic relationship in community...

    2007

  17. [25]

    , Baginski, H

    Metzler2022DetectingPH APACrefauthors Metzler, H. , Baginski, H. , Niederkrotenthaler, T. Garc \'i a, D. APACrefauthors \ 2022 . Detecting Potentially Harmful and Protective Suicide-Related Content on Twitter: Machine Learning Approach Detecting potentially harmful and protect...

    2022

  18. [26]

    , P \'e rez-Rosas, V

    Min2022PAIRPM APACrefauthors Min, D.J. , P \'e rez-Rosas, V. , Resnicow, K. Mihalcea, R. APACrefauthors \ 2022 . PAIR: Prompt-Aware margIn Ranking for Counselor Reflection Scoring in Motivational Interviewing Pair: Prompt-aware margin ranking for counselor reflection scoring i...

    2022

  19. [27]

    , Mikolov, T

    minaee2024survey APACrefauthors Minaee, S. , Mikolov, T. , Nikzad, N. , Chenaghlu, M. , Socher, R. , Amatriain, X. Gao, J. APACrefauthors \ 2024 . Large Language Models: A Survey Large language models: A survey . arXiv preprint arXiv:2402.06196 ,

    2024 arXiv

  20. [28]

    , Taher, M

    Nguyen2024DoLL APACrefauthors Nguyen, V.C. , Taher, M. , Hong, D. , Possobom, V.K. , Gopalakrishnan, V.T. , Raj, E. Choudhury, M.D. APACrefauthors \ 2024 . Do Large Language Models Align with Core Mental Health Counseling Competencies? Do large language models align with core ...

    2024 arXiv

  21. [29]

    , Resnicow, K

    perez2022pair APACrefauthors P \'e rez-Rosas, V. , Resnicow, K. , Mihalcea, R. \ . APACrefauthors \ 2022 . Pair: Prompt-aware margin ranking for counselor reflection scoring in motivational interviewing Pair: Prompt-aware margin ranking for counselor reflection scoring in moti...

    2022

  22. [30]

    , Zeng, Y

    qi2023fine APACrefauthors Qi, X. , Zeng, Y. , Xie, T. , Chen, P Y. , Jia, R. , Mittal, P. Henderson, P. APACrefauthors \ 2023 . Fine-tuning aligned language models compromises safety, even when users do not intend to! Fine-tuning aligned language models compromises safety, eve...

    2023 arXiv

  23. [31]

    Guidelines for Self-Promotion on Reddit

    reddit_selfpromotion APACrefauthors Reddit APACrefauthors \ 2025 1 . Guidelines for Self-Promotion on Reddit. Guidelines for self-promotion on reddit. https://www.reddit.com/wiki/selfpromotion . Accessed: 2025-05-19

    2025

  24. [32]

    reddit_vote_cheating APACrefauthors Reddit APACrefauthors \ 2025 2 . What Constitutes Vote Cheating and Vote Manipulation? What constitutes vote cheating and vote manipulation? https://www.reddit.com/r/reddit.com/wiki/faq/#wiki_what_constitutes_vote_cheating_and_vote_manipulat...

    2025

  25. [33]

    APACrefauthors \ 1957

    rogers1957necessary APACrefauthors Rogers, C.R. APACrefauthors \ 1957 . The necessary and sufficient conditions of therapeutic personality change. The necessary and sufficient conditions of therapeutic personality change. Journal of consulting psychology 21 2 95,

    1957

  26. [34]

    , Surana, H

    roy2025diagnostic APACrefauthors Roy, K. , Surana, H. , Eswaramoorthi, D. , Zi, Y. , Palit, V. , Garimella, R. Sheth, A. APACrefauthors \ 2025 . Large Language Models for Mental Health Diagnostic Assessments: Exploring The Potential of Large Language Models for Assisting with ...

    2025 arXiv

  27. [35]

    , Churchill, V

    saha2025large APACrefauthors Saha, A. , Churchill, V. , Rodriguez, A.D. , Kursuncu, U. Idris, M.Y. APACrefauthors \ 2025 . Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AI Large language models for cancer...

    2025 arXiv

  28. [36]

    , Alaniz, S

    salewski2023context APACrefauthors Salewski, L. , Alaniz, S. , Rio-Torto, I. , Schulz, E. Akata, Z. APACrefauthors \ 2023 . In-context impersonation reveals large language models' strengths and biases In-context impersonation reveals large language models' strengths and biases...

    2023

  29. [37]

    , Das, D

    sellam2020bleurt APACrefauthors Sellam, T. , Das, D. Parikh, A.P. APACrefauthors \ 2020 . BLEURT: Learning robust metrics for text generation Bleurt: Learning robust metrics for text generation . arXiv preprint arXiv:2004.04696 ,

    2020 arXiv

  30. [38]

    , Maher, R

    sengupta2021genbit APACrefauthors Sengupta, K. , Maher, R. , Groves, D. Olieman, C. APACrefauthors \ 2021 . GenBiT: measure and mitigate gender bias in language datasets Genbit: measure and mitigate gender bias in language datasets . Microsoft Journal of Applied Research 16 63--71,

    2021

  31. [39]

    , Miner, A.S

    sharma2020computational APACrefauthors Sharma, A. , Miner, A.S. , Atkins, D.C. Althoff, T. APACrefauthors \ 2020 . A computational approach to understanding empathy expressed in text-based mental health support A computational approach to understanding empathy expressed in tex...

    2020 arXiv

  32. [40]

    \ De Choudhury, M

    sharma2018mental APACrefauthors Sharma, E. \ De Choudhury, M. APACrefauthors \ 2018 . Mental health support and its relationship to linguistic accommodation in online communities Mental health support and its relationship to linguistic accommodation in online communities . Pro...

    2018

  33. [41]

    , Larrison, C.R

    sinha2023comparing APACrefauthors Sinha, G.R. , Larrison, C.R. , Brooks, I. Kursuncu, U. APACrefauthors \ 2023 . Comparing naturalistic mental health expressions on student loan debts using reddit and twitter Comparing naturalistic mental health expressions on student loan deb...

    2023

  34. [42]

    APACrefauthors \ 2011

    sun2011meta APACrefauthors Sun, S. APACrefauthors \ 2011 . Meta-analysis of Cohen’s kappa Meta-analysis of cohen’s kappa . Health Services and Outcomes Research Methodology 11 145--163,

    2011

  35. [43]

    , Zeng, Q

    tan2024democratizing APACrefauthors Tan, Z. , Zeng, Q. , Tian, Y. , Liu, Z. , Yin, B. Jiang, M. APACrefauthors \ 2024 . Democratizing large language models via personalized parameter-efficient fine-tuning Democratizing large language models via personalized parameter-efficient...

    2024 arXiv

  36. [44]

    \ Ito, T.A

    tobias2021anxiety APACrefauthors Tobias, M.R. \ Ito, T.A. APACrefauthors \ 2021 . Anxiety increases sensitivity to errors and negative feedback over time Anxiety increases sensitivity to errors and negative feedback over time . Biological psychology 162 108092,

    2021

  37. [45]

    , Martin, L

    touvron2023llama APACrefauthors Touvron, H. , Martin, L. , Stone, K. , Albert, P. , Almahairi, A. , Babaei, Y. others APACrefauthors \ 2023 . Llama 2: Open foundation and fine-tuned chat models Llama 2: Open foundation and fine-tuned chat models . arXiv preprint arXiv:2307.09288 ,

    2023 arXiv

  38. [46]

    , Naylor, M

    Vajre2021PsychBERTAM APACrefauthors Vajre, V. , Naylor, M. , Kamath, U. Shehu, A. APACrefauthors \ 2021 . PsychBERT: A Mental Health Language Model for Social Media Mental Health Behavioral Analysis Psychbert: A mental health language model for social media mental health behav...

    2021

  39. [47]

    APACrefauthors \ 2015

    wampold2015important APACrefauthors Wampold, B.E. APACrefauthors \ 2015 . How important are the common factors in psychotherapy? An update How important are the common factors in psychotherapy? an update . World psychiatry 14 3 270--277,

    2015

  40. [48]

    APACrefauthors \ 2024

    wang2024testing APACrefauthors Wang, W. APACrefauthors \ 2024 . Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness Testing and evaluation of large language models: Correctness, non-toxicity, and fairness . arXiv preprint arXiv:2409.00551 ,

    2024 arXiv

  41. [49]

    APACrefauthors \ 1951

    welch1951comparison APACrefauthors Welch, B.L. APACrefauthors \ 1951 . On the comparison of several mean values: an alternative approach On the comparison of several mean values: an alternative approach . Biometrika 38 3/4 330--336,

    1951

  42. [50]

    , Cui, H

    xu2023knowledge APACrefauthors Xu, R. , Cui, H. , Yu, Y. , Kan, X. , Shi, W. , Zhuang, Y. Yang, C. APACrefauthors \ 2023 . Knowledge-infused prompting: Assessing and advancing clinical text data generation with large language models Knowledge-infused prompting: Assessing and a...

    2023 arXiv

  43. [51]

    , Zhang, T

    Yang2023MentaLLaMAIM APACrefauthors Yang, K. , Zhang, T. , Kuang, Z Z. , Xie, Q. Ananiadou, S. APACrefauthors \ 2023 . MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models Mentallama: Interpretable mental health analysis on social media w...

    2023

  44. [52]

    , Torshizi, P.G

    Yongsatianchot2023InvestigatingLL APACrefauthors Yongsatianchot, N. , Torshizi, P.G. Marsella, S. APACrefauthors \ 2023 . Investigating Large Language Models’ Perception of Emotion Using Appraisal Theory Investigating large language models’ perception of emotion using appraisa...

    2023

  45. [53]

    , Kishore, V

    zhang2019bertscore APACrefauthors Zhang, T. , Kishore, V. , Wu, F. , Weinberger, K.Q. Artzi, Y. APACrefauthors \ 2019 . Bertscore: Evaluating text generation with bert Bertscore: Evaluating text generation with bert . arXiv preprint arXiv:1904.09675 ,

    2019 arXiv

  46. [54]

    , Bosselut, A

    zhang2022greaselm APACrefauthors Zhang, X. , Bosselut, A. , Yasunaga, M. , Ren, H. , Liang, P. , Manning, C.D. Leskovec, J. APACrefauthors \ 2022 . Greaselm: Graph reasoning enhanced language models for question answering Greaselm: Graph reasoning enhanced language models for ...

    2022 arXiv

  47. [55]

    write newline

    " write newline " cite write " FUNCTION editor.postfix editor num.names #1 > "( )" "( )" if FUNCTION editor.trans.postfix editor num.names #1 > "( )" "( )" if FUNCTION trans.postfix translator num.names #1 > "( )" "( )" if FUNCTION authors.editors.reflist.apa5 'field := 'dot :...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.