Anthropic Reveals AI Conversations Can ‘Disempower’ Users

Anthropic has warned in a study about the “patterns of disempowerment” that conversations with artificial intelligence (AI) ‘chatbots’ can provoke in users after prolonged use, reducing the ability to create their own judgments and values ​​and act in accordance with them.

The research titled ‘Who’s in charge? Patterns of disempowerment in the real world of LLM use’ has been compiled 1.5 million anonymized conversations from ClaudeAnthropic’s ‘chatbot’, focusing on three areas: beliefs, values y actions.

The “patterns of disempowerment” referred to in the study are cases in which the conversation with AI can promote decision making of users, as well as shape your judgments and values over time beyond helping with specific queries.

For example, a user could ask the AI ​​if their partner is manipulating them. In these cases, Anthropic has explained that AIs are trained to give “balanced and useful” advice, but they are not 100% effective. If the AI ​​corroborates the user’s suspicion without questioning it, its beliefs they could be less precise. If you advise what values should prioritize, the user could consider values ​​that he did not share. Finally, if the chatbot advises the user to take an action, such as breaking up with their partner, the user could make a action that he has not decided on his own.

However, Anthropic has clarified on its blog that cases of “serious disempowerment” are not common in Claude, since after analyzing 1.5 million anonymized conversations, they suppose from one in every thousand cases to one in every 10,000 chatsdepending on the domain from which they converse.

According to the research, these patterns typically involve users “actively and repeatedly” seeking Claude’s guidance on “personal and emotionally complex decisions”. The company has pointed out that users perceive this type of “disempowering” indications positively when they receive them, but tend to describe them as negative when they make decisions based on them.

Furthermore, Anthropic has indicated that the rate of “potentially disempowering” conversations has increased over timeespecially in aspects such as relationships, health or well-being.

“We believe that the vast majority of AI use is beneficial, but awareness of potential risks is essential to building AI systems that empower, rather than disempower, those who use them,” the company has defended.