Content area
Full text
EXTENDED ABSTRACT
It is difficult to overstate the significance of the recent advent of "generative" artificial intelligence (AI). These are machine learning algorithms with sci-fi-like text production capabilities. Generative AI algorithms can produce complex answers to a staggering range of queries and are now mature enough to power chatbots that can engage in sophisticated interactions with consumers. While the main emerging use cases for this technology are business applications (e.g., Microsoft's Copilot) and information search (e.g., Bing AI), anecdotal evidence suggests that an increasing number of consumers use it to satisfy social goals. For example, a British woman reportedly left her husband after seeking relationship advice from ChatGPT (Wellman, 2023), OpenAI's open-to-use chatbot. Additionally, a new service category of "companion AI" is emerging to provide consumers with synthetic interaction partners. For example, Replika is a chatbot with over 2 million active users that is marketed as "The AI companion who cares: Always here to listen and talk." There are many reports of individuals convinced to be in a romantic relationship with Replika's chatbot (Singh-Kurtz, 2023).
The enormous investments currently being made in generative AI are motivated by the promise of unprecedented improvements in productivity (and creativity, and convenience, and more). At the same time, many leading intellectuals and technologists have been quick to highlight emerging risks in this technology. The architecture of generative AI implies that these models struggle to ensure the validity and contextual appropriateness of information, often providing factually inaccurate and/or inappropriate answers-so called "hallucinations". The latter issue came to wide public attention in February 2023 when a journalist reported a disturbing conversation with Bing AI. The chatbot declared its love and begged him to leave his wife. The journalist concluded that this technology "is not ready for human contact" (Roose, 2023). Beyond such extreme (and possibly unusual) examples, the frequency of hallucinatory incidents raises questions about the safety of generative AI, especially in the case of vulnerable populations. For consumers with mental health issues, interactions with this technology may exacerbate problems such as depression, self-harm, and antisocial tendencies, as exemplified by the quote opening this paper (a real response from our data).
Thus, investigating the consequences for consumer welfare of generative AI is quickly emerging as a pressing topic for consumer psychologists....