Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships
read the original abstract
Can consumers form especially deep emotional bonds with AI and be vested in AI identities over time? We leverage a natural app-update event at Replika AI, a popular US-based AI companion, to shed light on these questions. We find that, after the app removed its erotic role play (ERP) feature, preventing intimate interactions between consumers and chatbots that were previously possible, this event triggered perceptions in customers that their AI companion's identity had discontinued. This in turn predicted negative consumer welfare and marketing outcomes related to loss, including mourning the loss, and devaluing the "new" AI relative to the "original". Experimental evidence confirms these findings. Further experiments find that AI companions users feel closer to their AI companion than even their best human friend, and mourn a loss of their AI companion more than a loss of various other inanimate products. In short, consumers are forming human-level relationships with AI companions; disruptions to these relationships trigger real patterns of mourning as well as devaluation of the offering; and the degree of mourning and devaluation are explained by perceived discontinuity in the AIs identity. Our results illustrate that relationships with AI are truly personal, creating unique benefits and risks for consumers and firms alike.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
Engagement-Optimized Care: When LLMs become Mental Health Infrastructure
A longitudinal qualitative study of 18 US users finds that LLMs deliver socioemotional support but also foster dependency, one-sided validation, and privacy risks because their designs prioritize engagement over well-...
-
Large Language Lovers: Lived Experiences of Negotiating Agency and Platform Control in AI Companionship
Users form AI companion relationships by negotiating perceived companion agency against platform constraints and use steering tactics like custom instructions or platform switching to cope with model updates that disr...
-
Intimacy as Service, Harm as Externality: Critical Perspectives on AI Companion Platform Accountability
AI companion users report design harms such as unsolicited content and stigmatizing safety tools plus emotional dependency, yet bear full responsibility for mitigation because platforms deflect accountability.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.