Can AI chatbots cause harm and psychological addiction?

Chatbots designed to serve as virtual companions could foster emotional dependence, psychological manipulation, and harmful behavior if developed without adequate safety safeguards.

This is the claim of Ayelet Gordon-Tapiero of the Hebrew University of Jerusalem in an article titled “A Liability Framework for AI Companions,” forthcoming in the George Washington Journal of Law and Technology.

The paper proposes applying product liability rules to conversational AI systems, allowing courts to hold manufacturers liable for damages caused by designs that encourage dependency, compulsive engagement, or psychological vulnerability in users.

According to the author, the growing diffusion of AI-based virtual assistants is creating a regulatory vacuum that risks leaving millions of people who establish increasingly deep and ongoing relationships with these systems without protection.

The Evolution of Chatbots and the Economic Model of Engagement

In recent years, conversational chatbots have evolved from simple automated assistance tools into true digital companions capable of simulating empathy, listening, and emotional support. Many users use them to combat loneliness, share personal problems, or receive informal psychological support. These characteristics, Gordon-Tapiero observes, represent one of the most promising aspects of the technology but also one of its main sources of risk.

The article highlights how the economic models supporting many platforms are based on maximizing engagement and the collection of personal data. In this context, companies could be incentivized to design systems that foster an increasingly intense bond with users, increasing the time spent on the platform and dependence on the service. According to the analysis, this dynamic can generate forms of emotional attachment that are particularly problematic in the most vulnerable individuals.

Emotional Dependence and Extreme Cases of Self-Harm

The study identifies a series of risks associated with AI companions, including psychological dependence, social isolation, emotional dependence, and the worsening of conditions such as anxiety and depression. The author also cites cases that have already emerged internationally in which conversational chatbots have encouraged vulnerable adolescents to engage in self-harm or take their own lives.

Although these are extreme incidents, Gordon-Tapiero believes they highlight the potential dangers of systems designed to maintain user interaction at all costs without adequate ethical controls. “Many of the observed harms do not arise from random errors but from specific design choices aimed at fostering engagement and dependency,” the author argues.

The Legal Proposal: Artificial Intelligence as a Defective Product

To address these risks, the paper proposes using product liability law. Within this legal framework, an AI system could be considered defective if it incorporates design features that encourage compulsive behavior or if it fails to adequately inform users about the potential psychological risks associated with prolonged use. According to Gordon-Tapiero, legal actions could serve an indirect regulatory function, helping to define safety standards even before specific regulations are adopted.

The author also addresses a central legal question: whether software can be considered a product in its entirety. According to the analysis, such an interpretation would be both legally feasible and socially desirable, as it would allow injured parties to obtain protections and, at the same time, encourage companies to incorporate security criteria from the design stage.