The reviewed record of science sign in
Pith

arxiv: 2607.06371 · v1 · pith:MLWZCCQN · submitted 2026-07-07 · cs.HC

The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI Chatbots

Jabari Kwesi , Jiaxun Cao , Hailee Cunningham , Pardis Emami-Naeini This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 08:00 UTCglm-5.2pith:MLWZCCQNrecord.jsonopen to challenge →

classification cs.HC
keywords generative AIprivacy controlsemotional supportchatbothuman-computer interactionsecurity usabilityvignette studymental health
0
0 comments X

The pith

Simple deletion controls beat sophisticated privacy tech for AI emotional support

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks a straightforward question: when people turn to AI chatbots for emotional support, which privacy and security controls actually make them more willing to open up? Through a vignette study of 354 U.S. participants who already use generative AI chatbots for emotional support, the authors tested nine distinct privacy controls — from simple conversation deletion to technically complex options like local-only processing and model training opt-outs — across three emotional contexts (anxiety, depression, interpersonal tension). The central finding is counterintuitive: the simplest controls won decisively. Participants rated deletion-based controls (deleting a conversation or an entire account) as most likely to increase their willingness to engage, their sense of protection, and their confidence in the chatbot's helpfulness. Technically sophisticated controls performed significantly worse — local-only processing and model training opt-outs actually reduced willingness to engage compared to deletion. The paper explains this pattern through three structural gaps. The comprehension gap: users cannot form accurate mental models of how controls like local processing or training opt-outs actually work, leading to confusion and mistrust. The assurance gap: even when users understand a control (like deletion), they doubt the platform will actually honor it — over a third of participants expressed skepticism that deletion truly removes their data. The affective urgency gap: controls that add friction (like multifactor authentication) are recognized as protective but rejected because users in emotional distress need immediate access. Notably, the emotional context (anxiety vs. depression vs. interpersonal tension) had no significant effect on any of these patterns, suggesting the findings are robust across types of distress.

Core claim

The paper's central discovery is that user-facing privacy controls in AI chatbots succeed or fail based on three distinct, identifiable gaps between what a control promises and what users can comprehend, trust, or tolerate under emotional distress. Deletion controls dominate because they are comprehensible, feel reversible, and impose no friction — even though users doubt they truly work. Sophisticated controls fail not because they are technically inferior but because users cannot understand them, cannot verify them, or cannot bear the friction they impose when seeking emotional relief. The paper classifies the nine controls into three categories — preventive (limiting data creation),revers

What carries the argument

Three structural gaps — comprehension, assurance, and affective urgency — form the explanatory framework. Preventive controls (local-only processing, model training opt-out, memory toggle, anonymous chat, non-mandatory login) suffer from the comprehension gap: users lack accurate mental models of how these mechanisms affect their data. Reversibility controls (delete conversation, delete account & data) suffer from the assurance gap: users understand the promise but cannot verify execution. Protective controls (MFA, access/sharing controls) suffer from the affective urgency gap: users recognize protection but experience authentication barriers as prohibitively burdensome during emotionaldist

If this is right

  • Designers of AI chatbots used for emotional support should prioritize simple, outcome-framed deletion controls over technically sophisticated privacy mechanisms, since users understand and trust 'delete' far more than 'local-only processing' or 'training opt-out.'
  • The finding that MFA is perceived as protective but reduces willingness to engage suggests that authentication design for emotional-support contexts needs context-sensitive defaults — frictionless access during acute distress with optional hardening for users who fear physically proximate adversaries.
  • The assurance gap — where users understand deletion but doubt it works — implies that verifiable transparency mechanisms (showing what data exists before and after deletion) may be more important than adding new controls.
  • Policy frameworks that place responsibility on users to calibrate their own privacy settings (as in the federal AI policy approach cited) may fail systematically, since users demonstrably cannot understand or verify the controls they are asked tomanage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the comprehension gap generalizes beyond the nine tested controls, then any new privacy mechanism introduced into AI chatbots will face adoption resistance proportional to how difficult it is to explain in plain language — regardless of its technical strength.
  • The finding that emotional context had no effect on control preferences suggests that privacy control design for AI chatbots can be unified rather than context-specific, simplifying the design space considerably.
  • The coexistence of desire and doubt (participants who wanted to disclose more but doubted controls would protect them) implies a population of users currently under-disclosing to AI chatbots — representing lost therapeutic value that better assurance mechanisms couldunlock.
  • The paper's vignette methodology may systematically overstate the affective urgency gap, since calm survey respondents may not accurately simulate the urgency that reduces MFA tolerance during acute emotionaldistress.

Load-bearing premise

The study's central design premise is that vignette-based hypothetical responses — where calm participants imagine how they would behave when emotionally distressed — reliably predict real-world behavior in actual emotional crises. If participants cannot accurately simulate the urgency and reduced tolerance for friction that accompanies acute distress, then the magnitude of the affective urgency finding may be inflated.

What would settle it

If a real-world deployment study showed that users in acute emotional distress actually tolerate MFA or other friction-inducing controls at rates similar to their vignette responses, the affective urgency gap would need to be reconsidered. Conversely, if technically sophisticated controls like local-only processing were reframed in plain outcome language and subsequently performed comparably to deletion controls, the comprehension gap — not the controls themselves — would be confirmed as the primarybarrier.

Figures

Figures reproduced from arXiv: 2607.06371 by Hailee Cunningham, Jabari Kwesi, Jiaxun Cao, Pardis Emami-Naeini.

Figure 1
Figure 1. Figure 1: Distribution of GenAI chatbot platforms par￾ticipants reported using for emotional support conversa￾tions. General-purpose LLMs (brown) dominate; AI com￾panion platforms (peach) represent a secondary but mean￾ingful presence. Percentages exceed 100%, as 28.8% of par￾ticipants reported multiple platforms. Privacy Control β OR SE Significance Delete Conversation Ref. — — — Delete Account & Data 0.064 1.07 0.… view at source ↗
Figure 2
Figure 2. Figure 2: Availability of S&P controls across the six GenAI platforms most frequently used by participants for emotional support as of the August–September 2025 audit period, ordered by adoption rate. Checkmarks indicate presence; gray cells indicate absence. • Secure container framing, wherein a majority of partic￾ipants viewed authentication as infrastructure that gates unauthorized access; the control very clearl… view at source ↗
read the original abstract

Chatbots powered by generative AI (e.g., OpenAI's ChatGPT and Google's Gemini) are increasingly being appropriated for emotional support and companionship. These tools offer a suite of security and privacy (S&P) controls, including model training opt-outs and memory toggles, yet how the presence of these controls influences users' attitudes toward emotionally sensitive disclosure remains understudied. We conducted a mixed-methods vignette study with 354 U.S. participants to examine how S&P controls influence users' willingness to engage with generative AI chatbots for emotional support, their perceptions of how protected they are when using these systems, and their perceptions of how effective the chatbots are for providing support. Controls enabling deletion of disclosures had the largest positive impact: these offerings outperformed technically sophisticated controls such as local-only processing and model training opt-outs, where participants expressed difficulty understanding the underlying mechanisms. Yet trust remains fragile, and participants often doubted S&P controls would function as promised. We conclude with actionable recommendations informed by our results to bridge users' comprehension gaps, build credible assurances, and properly calibrate barriers for users in distress.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 6 minor

Summary. This paper investigates how nine user-facing security and privacy (S&P) controls—derived from a systematic audit of 87 generative AI chatbot applications—influence users' willingness to engage, perceived protection, and perceived efficacy when using GenAI chatbots for emotional support. Using a mixed-methods vignette study (N=354 U.S. participants), the authors fit cumulative link mixed models (CLMMs) with participant random intercepts and supplement quantitative findings with inductive thematic analysis of open-ended responses. The central finding is that deletion-based controls dominate user preferences, significantly outperforming technically sophisticated controls like local-only processing and model training opt-outs, which suffer from comprehension gaps. The authors synthesize their findings into a three-gap framework (comprehension, assurance, affective urgency) and offer design and policy recommendations.

Significance. The paper addresses a timely and important gap at the intersection of usable privacy, conversational AI, and mental health. The saturation-based audit methodology is well-executed and follows HCI norms, and the nine-control taxonomy is derived independently rather than from prior theoretical commitments, avoiding circularity. The CLMMs are appropriate for the ordinal repeated-measures design, and the authors apply both BH FDR and Bonferroni corrections. The mixed-methods integration is a strength: qualitative codes provide mechanistic explanations for quantitative patterns (e.g., MFA's dissociation between protection and willingness). The three-gap framework is actionable and falsifiable. The vignette-behavior gap is acknowledged in §3.5 and is standard for HCI vignette methodology. Open science practices are noted, with survey instruments and analysis code provided in a supplementary repository.

major comments (2)
  1. [§3.2, Experimental Design] The between-subjects assignment of Context of Disclosure is based on participants' highest-rated use case rather than random assignment. This self-selection mechanism means the null finding for context (Tables 3–5, Table 7: Depression and Interpersonal Tension coefficients near zero, all p > .05) could be confounded. Participants who select 'Anxiety & Stress' may differ systematically from those who select 'Interpersonal Tension' in ways that mask context effects. The paper claims context 'had no significant effect on user perceptions' (§4), but this is a secondary claim that the design cannot strongly support. The authors should explicitly acknowledge this confound when interpreting the null context effect, or soften the claim to note that context invariance is observed only among self-selected context assignments. This does not undermine the central within-subjects finding about deleti
  2. [§5.4, Policy Implications] The policy discussion states: 'Our findings suggest the empirical ground favors the latter orientation' (referring to California's platform-level vetting approach over the federal user-responsibility model). This claim overreaches the evidence. The study measures user perceptions of S&P controls in a hypothetical vignette context; it does not evaluate the effectiveness of regulatory frameworks or platform-level vetting. The finding that users lack comprehension of certain controls supports the general observation that user-responsibility models face challenges, but it does not constitute empirical evidence favoring one regulatory approach over another. The authors should reframe this as suggesting that their findings raise concerns about user-responsibility models, rather than claiming the empirical ground favors a specific regulatory orientation.
minor comments (6)
  1. [§3.2] The paper states each participant evaluated 4 of 9 S&P controls. Given the within-subjects design, it would be helpful to report how many participants were exposed to each control and whether any control was systematically under- or over-represented due to the randomization scheme.
  2. [§3.5] The vignette-behavior gap limitation is acknowledged but could be strengthened. The Baruh et al. meta-analysis citation is somewhat general; the authors could note that their vignettes are contextually specific (as Baruh et al. recommend), which is a design strength that partially mitigates the gap.
  3. [Table 7 note] The rationale for retaining only race/ethnicity as a demographic covariate is briefly mentioned ('covariate selection rationale'). A more explicit justification for why other demographics (e.g., age, gender, mental health care history) were removed would improve transparency.
  4. [Figure 2] The categorization of controls into preventive, reversibility, and protective is introduced in the Discussion (§5.1–§5.3) but not in the Methods or Results. A brief note in the Methods explaining that this taxonomy emerged from the qualitative analysis would help readers understand its provenance.
  5. [Ethical Considerations] The compensation ($2.50 for ~17 minutes) is noted as consistent with Prolific norms. This is acceptable, but the paper could note the hourly rate (~$8.82/hr) for reader expectations.
  6. [§4.1] The phrase 'betrayal framing' is used in the section heading but not clearly defined in the body text. A brief operational definition would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive review. Both major comments identify legitimate issues that we will address in revision. Below we respond to each point.

read point-by-point responses

  1. Referee: The between-subjects assignment of Context of Disclosure is based on participants' highest-rated use case rather than random assignment. This self-selection mechanism means the null finding for context could be confounded. The authors should explicitly acknowledge this confound or soften the claim.

    Authors: The referee is correct. Context of Disclosure was assigned based on each participant's highest-rated use case in a pre-task, not by random assignment. This means participants self-selected into contexts, and systematic differences between groups (e.g., participants who rate 'Anxiety & Stress' highest may differ in clinically relevant ways from those who rate 'Interpersonal Tension' highest) could mask true context effects. Our design cannot rule out this confound. We chose self-selected context assignment to maximize ecological validity—participants evaluated vignettes in the context most relevant to them—but this design decision trades internal validity for contextual relevance, and we should have been more transparent about the tradeoff. In revision, we will (1) add an explicit acknowledgment of the self-selection confound in §3.5 (Limitations), noting that the null context effect is observed only among self-selected context assignments and cannot be interpreted as evidence of true context invariance, and (2) soften the claim in §4 from 'context had no significant effect on user perceptions' to language specifying that 'no significant differences were observed across self-selected disclosure contexts.' We will also note that a fully randomized design would be needed to draw stronger conclusions about context effects. We agree this does not undermine the central within-subjects finding about deletion controls, which is independent of the context assignment mechanism. revision: yes

  2. Referee: The policy discussion states 'Our findings suggest the empirical ground favors the latter orientation' (California's platform-level vetting over the federal user-responsibility model). This overreaches the evidence, as the study measures user perceptions in a hypothetical vignette context, not the effectiveness of regulatory frameworks.

    Authors: The referee is right that our study does not evaluate regulatory frameworks or platform-level vetting. Our evidence is about user comprehension of and trust in S&P controls, not about the comparative effectiveness of regulatory approaches. The sentence as written conflates 'our findings raise concerns about user-responsibility models' with 'the empirical ground favors a specific regulatory orientation,' which is a stronger claim than our data support. In revision, we will reframe this passage to state that our findings raise concerns about user-responsibility models—specifically, that such models presuppose a deliberative capacity to understand and calibrate S&P controls that our participants often lacked—without claiming that the empirical ground favors any particular regulatory framework. We will also clarify that we offer our three-gap framework as assessment criteria that regulators may find useful, not as empirical validation of any specific regulatory approach. revision: yes

Circularity Check

0 steps flagged

No significant circularity; one minor self-citation for 'intangible vulnerability' concept that is not load-bearing for the central findings.

full rationale

The paper's central empirical findings derive from an independent audit of 87 apps (producing the nine-control taxonomy), a pre-registered vignette survey (N=354), and inductively coded qualitative responses. The quantitative results (CLMM coefficients in Tables 3-5) are fitted from participant survey data, not from prior theoretical commitments. The qualitative codebook was developed inductively from 10% of responses. The three structural gaps (comprehension, assurance, affective urgency) are interpretive syntheses of the qualitative data, not definitions that presuppose the quantitative outcomes. The one self-citation is to Kwesi et al. [43] for the 'intangible vulnerability' concept in §1, which provides background framing but is not load-bearing for any of the paper's statistical results or design recommendations. The vignette-behavior gap acknowledged in §3.5 is a standard methodological limitation, not a circularity. No step in the derivation chain reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities, particles, forces, or mathematical objects. The three-gap framework (comprehension gap, assurance gap, affective urgency gap) is a conceptual organizing structure, not an invented entity. The nine-control taxonomy is an empirical classification derived from observation, not a postulated construct. No free parameters are fitted by hand or introduced ad hoc; the CLMM coefficients are estimated from data via standard maximum likelihood.

axioms (5)
  • domain assumption Vignette-based hypothetical responses reliably predict real-world behavior in emotionally distressed states
    The entire experimental design rests on participants accurately simulating how they would respond to S&P controls during emotional distress. Acknowledged as a limitation in §3.5 but structurally load-bearing for all findings.
  • domain assumption Saturation-based sampling of 87 apps from a 344-app corpus produces a representative taxonomy of S&P controls
    The nine-control taxonomy depends on the claim that auditing the top 10% by popularity plus three random 5% batches captures all relevant controls. The second and third batches yielded no novel controls, but this only confirms saturation within the sampled frame.
  • domain assumption Participants' self-reported use of GenAI chatbots for emotional support at least monthly defines a population whose perceptions are relevant to the research questions
    The screening criterion (§3.2) excludes non-users and active avoiders, whose privacy concerns may differ systematically. This is a boundary condition on generalizability.
  • domain assumption The three emotional contexts (anxiety, depression, interpersonal tension) are representative of the primary use cases for GenAI emotional support
    Selected from prior psychotherapy literature (§3.2) and supported by recent survey data, but the specific three-context selection constrains the generalizability of the 'no context effect' finding.
  • standard math Cumulative link mixed models with the specified fixed and random effects structure are the appropriate statistical framework for ordinal vignette ratings
    CLMMs with participant random intercepts are a standard approach for ordinal repeated-measures data, well-justified by Christensen and Taylor et al. citations.

pith-pipeline@v1.1.0-glm · 29474 in / 3192 out tokens · 396599 ms · 2026-07-08T08:00:45.340559+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

93 extracted references · 93 canonical work pages · 8 internal anchors

  1. [1]

    A., ALAJLANI, M., ALALWAN, A

    ABD-ALRAZAQ, A. A., ALAJLANI, M., ALALWAN, A. A., BE- WICK, B. M., GARDNER, P.,ANDHOUSEH, M. An overview of the features of chatbots in mental health: A scoping review.Interna- tional journal of medical informatics 132(2019), 103978

    work page 2019

  2. [2]

    Pri- vacy and human behavior in the age of information.Science 347, 6221 (2015), 509–514

    ACQUISTI, A., BRANDIMARTE, L.,ANDLOEWENSTEIN, G. Pri- vacy and human behavior in the age of information.Science 347, 6221 (2015), 509–514

    work page 2015

  3. [3]

    Privacy attitudes and privacy behavior: Losses, gains, and hyperbolic discounting

    ACQUISTI, A.,ANDGROSSKLAGS, J. Privacy attitudes and privacy behavior: Losses, gains, and hyperbolic discounting. InEconomics of information security. Springer, 2004, pp. 165–178

    work page 2004

  4. [4]

    Privacy and rationality in indi- vidual decision making.IEEE security & privacy 3, 1 (2005), 26–33

    ACQUISTI, A.,ANDGROSSKLAGS, J. Privacy and rationality in indi- vidual decision making.IEEE security & privacy 3, 1 (2005), 26–33

    work page 2005

  5. [5]

    AMA urges congress to strengthen safeguards for AI chatbots

    AMERICANMEDICALASSOCIATION. AMA urges congress to strengthen safeguards for AI chatbots. AMA Press Release, Apr. 2026

    work page 2026

  6. [6]

    Protecting the wellbeing of our users

    ANTHROPIC. Protecting the wellbeing of our users. Anthropic News, 2025

    work page 2025

  7. [7]

    Estimating survey fatigue in time use study

    BACKOR, K., GOLDE, S.,ANDNIE, N. Estimating survey fatigue in time use study. Ininternational association for time use research conference. Washington, DC(2007)

    work page 2007

  8. [8]

    Methods for the synthesis of qualitative research: a critical review.BMC medical research method- ology 9, 1 (2009), 59

    BARNETT-PAGE, E.,ANDTHOMAS, J. Methods for the synthesis of qualitative research: a critical review.BMC medical research method- ology 9, 1 (2009), 59

    work page 2009

  9. [9]

    Online privacy concerns and privacy management: A meta-analytical review.Journal of Communication 67, 1 (2017), 26–53

    BARUH, L., SECINTI, E.,ANDCEMALCILAR, Z. Online privacy concerns and privacy management: A meta-analytical review.Journal of Communication 67, 1 (2017), 26–53

    work page 2017

  10. [10]

    Ai chatbots upended their lives

    BOND, S. Ai chatbots upended their lives. they found support from each other. NPR, Jan. 2026

    work page 2026

  11. [11]

    Mis- placed confidences: Privacy and the control paradox.Social psycho- logical and personality science 4, 3 (2013), 340–347

    BRANDIMARTE, L., ACQUISTI, A.,ANDLOEWENSTEIN, G. Mis- placed confidences: Privacy and the control paradox.Social psycho- logical and personality science 4, 3 (2013), 340–347

    work page 2013

  12. [12]

    P.,ANDANDERSON, D

    BURNHAM, K. P.,ANDANDERSON, D. R.Model selection and multimodel inference: a practical information-theoretic approach. Springer, 2002

    work page 2002

  13. [13]

    BURTON, C., SZENTAGOTAITATAR, A., MCKINSTRY, B., MATH- ESON, C., MATU, S., MOLDOVAN, R., MACNAB, M., FARROW, E., DAVID, D., PAGLIARI, C.,ET AL. Pilot randomised controlled trial of help4mood, an embodied virtual agent-based system to sup- port treatment of depression.Journal of telemedicine and telecare 22, 6 (2016), 348–355

    work page 2016

  14. [14]

    SB 243: Artificial Intelli- gence: Chatbots.https://leginfo.legislature.ca.gov/faces/ billNavClient.xhtml?bill_id=202520260SB243, 2025

    CALIFORNIASTATELEGISLATURE. SB 243: Artificial Intelli- gence: Chatbots.https://leginfo.legislature.ca.gov/faces/ billNavClient.xhtml?bill_id=202520260SB243, 2025. Enacted October 2025, effective January 1, 2026. Requires AI chatbots to pro- vide suicide prevention resources, prohibits chatbots from claiming to be human, and mandates disclosure that users...

    work page 2025

  15. [15]

    Extracting training data from large language models

    CARLINI, N., TRAMER, F., WALLACE, E., JAGIELSKI, M., HERBERT-VOSS, A., LEE, K., ROBERTS, A., BROWN, T., SONG, D., ERLINGSSON, U.,ET AL. Extracting training data from large language models. In30th USENIX security symposium (USENIX Se- curity 21)(2021), pp. 2633–2650

    work page 2021

  16. [16]

    CHRISTENSEN, R. H. B.ordinal—Regression Models for Ordinal Data, 2025

    work page 2025

  17. [17]

    AI mental health apps: Ratings and re- views

    COMMONSENSEMEDIA. AI mental health apps: Ratings and re- views. Common Sense Media AI Ratings, 2026

    work page 2026

  18. [18]

    M., BIYANOVA, T., ELHAI, J., SCHNURR, P

    COOK, J. M., BIYANOVA, T., ELHAI, J., SCHNURR, P. P.,AND COYNE, J. C. What do psychotherapists really do in practice? an internet study of over 2,000 practitioners.Psychotherapy: Theory, Research, Practice, Training 47, 2 (2010), 260

    work page 2010

  19. [19]

    DAS, S., WANG, B., TINGLE, Z.,ANDCAMP, L. J. Evaluating user perception of multi-factor authentication: A systematic review.arXiv preprint arXiv:1908.05901(2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1908

  20. [20]

    Benefits and Harms of Large Language Models in Digital Mental Health

    DECHOUDHURY, M., PENDSE, S. R.,ANDKUMAR, N. Benefits and harms of large language models in digital mental health.arXiv preprint arXiv:2311.14693(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    A.,ANDTUROW, J

    DRAPER, N. A.,ANDTUROW, J. The corporate cultivation of digital resignation.New media & society 21, 8 (2019), 1824–1839

    work page 2019

  22. [22]

    Beyond data pri- vacy: New privacy risks for large language models.arXiv preprint arXiv:2509.14278(2025)

    DU, Y., LI, Z., LI, N.,ANDDING, B. Beyond data pri- vacy: New privacy risks for large language models.arXiv preprint arXiv:2509.14278(2025)

    work page arXiv 2025

  23. [23]

    First I “like” it, then I hide it: Folk theories of social feeds

    ESLAMI, M., KARAHALIOS, K., SANDVIG, C., VACCARO, K., RICKMAN, A., HAMILTON, K.,ANDKIRLIK, A. First I “like” it, then I hide it: Folk theories of social feeds. InProceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16)(2016), ACM, pp. 2371–2382

    work page 2016

  24. [24]

    Article 12: Transparent information, communication and modalities for the exercise of the rights of the data subject

    EUROPEANPARLIAMENT ANDCOUNCIL OF THEEUROPEAN UNION. Article 12: Transparent information, communication and modalities for the exercise of the rights of the data subject. General Data Protection Regulation (EU) 2016/679, 2016

    work page 2016

  25. [25]

    Legislative recommenda- tions: A national policy framework for artificial intelligence

    EXECUTIVEOFFICE OF THEPRESIDENT. Legislative recommenda- tions: A national policy framework for artificial intelligence. White House, Mar. 2026

    work page 2026

  26. [26]

    K., DARCY, A.,ANDVIERHILE, M

    FITZPATRICK, K. K., DARCY, A.,ANDVIERHILE, M. Delivering cognitive behavior therapy to young adults with symptoms of depres- sion and anxiety using a fully automated conversational agent (woe- bot): a randomized controlled trial.JMIR mental health 4, 2 (2017), e7785

    work page 2017

  27. [27]

    a stalker’s paradise

    FREED, D., PALMER, J., MINCHALA, D., LEVY, K., RISTENPART, T.,ANDDELL, N. “a stalker’s paradise” how intimate partner abusers exploit technology. InProceedings of the 2018 CHI conference on human factors in computing systems(2018), pp. 1–13

    work page 2018

  28. [28]

    OpenAI launches ChatGPT health in a push to become a hub for personal health data

    GOLDMAN, S. OpenAI launches ChatGPT health in a push to become a hub for personal health data. Fortune, Jan. 2026

    work page 2026

  29. [29]

    GREEN, P.,ANDMACLEOD, C. J. Simr: An r package for power analysis of generalized linear mixed models by simulation.Methods in Ecology and Evolution 7, 4 (2016), 493–498

    work page 2016

  30. [30]

    A simple method to assess and report thematic saturation in qualitative research.PloS one 15, 5 (2020), e0232076

    GUEST, G., NAMEY, E.,ANDCHEN, M. A simple method to assess and report thematic saturation in qualitative research.PloS one 15, 5 (2020), e0232076

    work page 2020

  31. [31]

    H., FARRINGTON, J., KEEN, T., LI, K.,ET AL

    GUO, Z., LAI, A., THYGESEN, J. H., FARRINGTON, J., KEEN, T., LI, K.,ET AL. Large language models for mental health applications: systematic review.JMIR mental health 11, 1 (2024), e57400

    work page 2024

  32. [32]

    E.Regression Modeling Strategies: With Applica- tions to Linear Models, Logistic and Ordinal Regression, and Survival Analysis, 2 ed

    HARRELL, JR., F. E.Regression Modeling Strategies: With Applica- tions to Linear Models, Logistic and Ordinal Regression, and Survival Analysis, 2 ed. Springer Series in Statistics. Springer, Cham, Switzer- land, 2015

    work page 2015

  33. [33]

    HUA, Y., SIDDALS, S., MA, Z., GALATZER-LEVY, I., XIA, W., HAU, C., NA, H., FLATHERS, M., LINARDON, J., AYUBCHA, C., ET AL. Charting the evolution of artificial intelligence mental health chatbots from rule-based systems to large language models: a system- atic review.World Psychiatry 24, 3 (2025), 383–394

    work page 2025

  34. [34]

    Challenges in building intelli- gent open-domain dialog systems.ACM Transactions on Information Systems (TOIS) 38, 3 (2020), 1–32

    HUANG, M., ZHU, X.,ANDGAO, J. Challenges in building intelli- gent open-domain dialog systems.ACM Transactions on Information Systems (TOIS) 38, 3 (2020), 1–32

    work page 2020

  35. [35]

    HB 1806: Wellness and Oversight for Psychological Resources Act.https://www.billtrack50.com/ billdetail/1805267, 2025

    ILLINOISGENERALASSEMBLY. HB 1806: Wellness and Oversight for Psychological Resources Act.https://www.billtrack50.com/ billdetail/1805267, 2025. Enacted 2025. Prohibits artificial intel- ligence from providing mental health treatment in Illinois. Violations subject to fines up to $10,000

    work page arXiv 2025

  36. [36]

    An empathy- driven, conversational artificial intelligence agent (wysa) for digital mental well-being: real-world data evaluation mixed-methods study

    INKSTER, B., SARDA, S., SUBRAMANIAN, V.,ET AL. An empathy- driven, conversational artificial intelligence agent (wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR mHealth and uHealth 6, 11 (2018), e12106

    work page 2018

  37. [37]

    Rethinking Large Language Models in Mental Health Applications

    JI, S., ZHANG, T., YANG, K., ANANIADOU, S.,ANDCAMBRIA, E. Rethinking large language models in mental health applications. arXiv preprint arXiv:2311.11267(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    JIBINJOSEPH. Be careful with meta ai: You might accidentally make your chats public.https://www.pcmag.com/news/be- careful- with- meta- ai- you- might- accidentally- make- your- chats- public, June 2025

    work page 2025

  39. [39]

    Demand for 988 continues to grow at third anniversary

    KAISERFAMILYFOUNDATION. Demand for 988 continues to grow at third anniversary. KFF, July 2025

    work page 2025

  40. [40]

    My data just goes everywhere:

    KANG, R., DABBISH, L., FRUCHTER, N.,ANDKIESLER, S. “My data just goes everywhere:” User mental models of the Internet and implications for privacy and security. InProceedings of the Eleventh Symposium On Usable Privacy and Security (SOUPS 2015)(Ottawa, Canada, 2015), USENIX Association, pp. 39–52

    work page 2015

  41. [41]

    User privacy and large language models: An analysis of frontier devel- opers’ privacy policies

    KING, J., KLYMAN, K., CAPSTICK, E., SAADE, T.,ANDHSIEH, V. User privacy and large language models: An analysis of frontier devel- opers’ privacy policies. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society(2025), vol. 8, pp. 1465–1477

    work page 2025

  42. [42]

    R., VIDGEN, B., RÖTTGER, P.,ANDHALE, S

    KIRK, H. R., VIDGEN, B., RÖTTGER, P.,ANDHALE, S. A. The benefits, risks and bounds of personalizing the alignment of large lan- guage models to individuals.Nature Machine Intelligence 6, 4 (2024), 383–392

    work page 2024

  43. [43]

    Exploring user security and privacy attitudes and concerns to- ward the use of general-purpose llm chatbots for mental health

    KWESI, J., CAO, J., MANCHANDA, R.,ANDEMAMI-NAEINI, P. Exploring user security and privacy attitudes and concerns to- ward the use of general-purpose llm chatbots for mental health. In 34th USENIX Security Symposium (USENIX Security 25)(2025), pp. 6007–6024

    work page 2025

  44. [44]

    LAESTADIUS, L., BISHOP, A., GONZALEZ, M., ILLEN ˇCÍK, D., ANDCAMPOS-CASTILLO, C. Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot replika.New Media & Society 26, 10 (2024), 5923–5941

    work page 2024

  45. [45]

    LEDBETTER, A. M. Measuring online communication attitude: In- strument development and validation.Communication Monographs 76, 4 (2009), 463–486

    work page 2009

  46. [46]

    i hear you, i feel you

    LEE, Y.-C., YAMASHITA, N., HUANG, Y.,ANDFU, W. " i hear you, i feel you": encouraging deep self-disclosure through a chatbot. InProceedings of the 2020 CHI conference on human factors in com- puting systems(2020), pp. 1–12

    work page 2020

  47. [47]

    E.,ANDMOHR, D

    LI, H., ZHANG, R., LEE, Y.-C., KRAUT, R. E.,ANDMOHR, D. C. Systematic review and meta-analysis of ai-based conversational agents for promoting mental health and well-being.NPJ Digital Medicine 6, 1 (2023), 236

    work page 2023

  48. [48]

    The diverse domains of quantified selves: self-tracking modes and dataveillance.Economy and Society 45, 1 (2016), 101– 122

    LUPTON, D. The diverse domains of quantified selves: self-tracking modes and dataveillance.Economy and Society 45, 1 (2016), 101– 122

    work page 2016

  49. [49]

    Self-control in cyberspace: Applying dual systems theory to a review of digital self- control tools

    LYNGS, U., LUKOFF, K., SLOVAK, P., BINNS, R., SLACK, A., IN- ZLICHT, M., VANKLEEK, M.,ANDSHADBOLT, N. Self-control in cyberspace: Applying dual systems theory to a review of digital self- control tools. Inproceedings of the 2019 CHI conference on human factors in computing systems(2019), pp. 1–18

    work page 2019

  50. [50]

    Understanding the benefits and chal- lenges of using large language model-based conversational agents for mental well-being support

    MA, Z., MEI, Y.,ANDSU, Z. Understanding the benefits and chal- lenges of using large language model-based conversational agents for mental well-being support. InAMIA Annual Symposium Proceed- ings(2023), vol. 2023, American Medical Informatics Association, p. 1105

    work page 2023

  51. [51]

    MCDONALD, N., SCHOENEBECK, S.,ANDFORTE, A. Reliability and inter-rater reliability in qualitative research: Norms and guide- lines for cscw and hci practice.Proceedings of the ACM on human- computer interaction 3, CSCW (2019), 1–23

    work page 2019

  52. [52]

    Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory

    MIRESHGHALLAH, N., KIM, H., ZHOU, X., TSVETKOV, Y., SAP, M., SHOKRI, R.,ANDCHOI, Y. Can llms keep a secret? testing pri- vacy implications of language models via contextual integrity theory. arXiv preprint arXiv:2310.17884(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  53. [53]

    Digi- tal identity guidelines: Authentication and authenticator management

    NATIONALINSTITUTE OFSTANDARDS ANDTECHNOLOGY. Digi- tal identity guidelines: Authentication and authenticator management. NIST Special Publication 800-63B-4, U.S. Department of Commerce, Gaithersburg, MD, 2025

    work page 2025

  54. [54]

    NEDERHOF, A. J. Methods of coping with social desirability bias: A review.European journal of social psychology 15, 3 (1985), 263–280

    work page 1985

  55. [55]

    AB 406: Mental Health Services, 2025

    NEVADALEGISLATURE. AB 406: Mental Health Services, 2025. En- acted 2025. Prohibits AI systems from independently providing men- tal health diagnosis or treatment in Nevada. Requires human oversight for any AI-assisted mental health services

    work page 2025

  56. [56]

    C., KADHE, S

    NGONG, I. C., KADHE, S. R., WANG, H., MURUGESAN, K., WEISZ, J. D., DHURANDHAR, A.,ANDRAMAMURTHY, K. N. Pro- tecting users from themselves: Safeguarding contextual privacy in in- teractions with conversational agents. InFindings of the Association for Computational Linguistics: ACL 2025(2025), pp. 26196–26220

    work page 2025

  57. [57]

    Privacy as contextual integrity.Washington Law Review 79, 1 (2004), 119–158

    NISSENBAUM, H. Privacy as contextual integrity.Washington Law Review 79, 1 (2004), 119–158

    work page 2004

  58. [58]

    A., HORNE, D

    NORBERG, P. A., HORNE, D. R.,ANDHORNE, D. A. The privacy paradox: Personal information disclosure intentions versus behaviors. Journal of consumer affairs 41, 1 (2007), 100–126

    work page 2007

  59. [59]

    Executive order N-5-26: Trusted AI procurement

    OFFICE OFGOVERNORGAVINNEWSOM. Executive order N-5-26: Trusted AI procurement. State of California, Office of the Governor, Mar. 2026. Signed March 30, 2026

    work page 2026

  60. [60]

    Introducing ChatGPT health.https://openai.com/ index/introducing-chatgpt-health/, Jan

    OPENAI. Introducing ChatGPT health.https://openai.com/ index/introducing-chatgpt-health/, Jan. 2026

    work page 2026

  61. [61]

    ORNE, M. T. Demand characteristics and the concept of quasi- controls.Artifacts in behavioral research: Robert Rosenthal and Ralph L. Rosnow’s classic books 110(2009), 110–137

    work page 2009

  62. [62]

    ORTLOFF, A.-M., FASSL, M., PONTICELLO, A., MARTIUS, F., MERTENS, A., KROMBHOLZ, K.,ANDSMITH, M. Different re- searchers, different results? analyzing the influence of researcher ex- perience and data type during qualitative analysis of an interview and survey study on security advice. InProceedings of the 2023 CHI Con- ference on Human Factors in Computin...

    work page 2023

  63. [63]

    Prolific

    PALAN, S.,ANDSCHITTER, C. Prolific. ac—a subject pool for on- line experiments.Journal of behavioral and experimental finance 17 (2018), 22–27

    work page 2018

  64. [64]

    Exploring relationship development with social chatbots: A mixed-method study of replika

    PENTINA, I., HANCOCK, T.,ANDXIE, T. Exploring relationship development with social chatbots: A mixed-method study of replika. Computers in Human Behavior 140(2023), 107600

    work page 2023

  65. [65]

    E., LOCKARD, A

    PÉREZ-ROJAS, A. E., LOCKARD, A. J., BARTHOLOMEW, T. T., JANIS, R. A., CARNEY, D. M., XIAO, H., YOUN, S. J., SCOFIELD, B. E., LOCKE, B. D., CASTONGUAY, L. G.,ET AL. Presenting con- cerns in counseling centers: The view from clinicians on the ground. Psychological Services 14, 4 (2017), 416

    work page 2017

  66. [66]

    Ai chatbots and challenges of hipaa com- pliance for ai developers and vendors.Journal of Law, Medicine & Ethics 51, 4 (2023), 988–995

    REZAEIKHONAKDAR, D. Ai chatbots and challenges of hipaa com- pliance for ai developers and vendors.Journal of Law, Medicine & Ethics 51, 4 (2023), 988–995

    work page 2023

  67. [67]

    Large lan- guage models as mental health resources: Patterns of use in the united states.Practice Innovations(2025)

    ROUSMANIERE, T., ZHANG, Y., LI, X.,ANDSHAH, S. Large lan- guage models as mental health resources: Patterns of use in the united states.Practice Innovations(2025)

    work page 2025

  68. [68]

    SAGE Publications, London, 2021

    SALDAÑA, J.The Coding Manual for Qualitative Researchers, 4th ed. SAGE Publications, London, 2021

    work page 2021

  69. [69]

    SoK: The privacy paradox of large language models: Advancements, privacy risks, and mitigation

    SHANMUGARASA, Y., DING, M., CHAMIKARA, M.,ANDRAKO- TOARIVELO, T. SoK: The privacy paradox of large language models: Advancements, privacy risks, and mitigation. InProceedings of the 20th ACM Asia Conference on Computer and Communications Secu- rity (ASIA CCS ’25)(Hanoi, Vietnam, 2025), ACM

    work page 2025

  70. [70]

    Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction

    SHARMA, A., RUSHTON, K., LIN, I. W., WADDEN, D., LUCAS, K. G., MINER, A. S., NGUYEN, T.,ANDALTHOFF, T. Cognitive reframing of negative thoughts through human-language model inter- action.arXiv preprint arXiv:2305.02466(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  71. [71]

    L., HAMILTON-PAGE, M., JADAD, A

    SHEN, N., LEVITAN, M.-J., JOHNSON, A., BENDER, J. L., HAMILTON-PAGE, M., JADAD, A. A. R.,ANDWILJER, D. Finding a depression app: a review and content analysis of the depression app marketplace.JMIR mHealth and uHealth 3, 1 (2015), e3713

    work page 2015

  72. [72]

    I.,AND BRANDTZAEG, P

    SKJUVE, M., FØLSTAD, A., FOSTERVOLD, K. I.,AND BRANDTZAEG, P. B. My chatbot companion-a study of human- chatbot relationships.International Journal of Human-Computer Studies 149(2021), 102601

    work page 2021

  73. [73]

    SOLOVE, D. J. The myth of the privacy paradox.Geo. Wash. L. Rev. 89(2021), 1

    work page 2021

  74. [74]

    The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support

    SONG, I., PENDSE, S. R., KUMAR, N.,ANDDECHOUDHURY, M. The typing cure: Experiences with large language model chatbots for mental health support.arXiv preprint arXiv:2401.14362(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  75. [75]

    Progressive disclosure: When, why, and how do users want algorithmic transparency information? ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 4 (2020), 1–32

    SPRINGER, A.,ANDWHITTAKER, S. Progressive disclosure: When, why, and how do users want algorithmic transparency information? ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 4 (2020), 1–32

    work page 2020

  76. [76]

    Be- yond memorization: Violating privacy via inference with large lan- guage models

    STAAB, R., VERO, M., BALUNOVIC, M.,ANDVECHEV, M. Be- yond memorization: Violating privacy via inference with large lan- guage models. InThe Twelfth International Conference on Learning Representations(2024)

    work page 2024

  77. [77]

    L.,ANDBLANDFORD, A

    STAWARZ, K., COX, A. L.,ANDBLANDFORD, A. Beyond self- tracking and reminders: designing smartphone apps that support habit formation. InProceedings of the 33rd annual ACM conference on human factors in computing systems(2015), pp. 2653–2662

    work page 2015

  78. [78]

    L.,ANDCORBIN, J

    STRAUSS, A. L.,ANDCORBIN, J. M.Basics of Qualitative Re- search: Grounded Theory Procedures and Techniques. SAGE Publi- cations, Newbury Park, CA, 1990

    work page 1990

  79. [79]

    E., ROUSSELET, G

    TAYLOR, J. E., ROUSSELET, G. A., SCHEEPERS, C.,ANDSERENO, S. C. Rating norms should be calculated from cumulative link mixed effects models.Behavior Research Methods 55, 5 (2023), 2175–2196

    work page 2023

  80. [80]

    Google and chatbot start-up character move to settle teen suicide lawsuits

    TIKU, N. Google and chatbot start-up character move to settle teen suicide lawsuits. The Washington Post, Jan. 2026

    work page 2026

Showing first 80 references.