Anthropic Updates Policy to Prohibit Abusive Behavior Toward AI Models

Anthropic updated its usage policy to include a prohibition on sustained and needless abusive or cruel behavior toward its AI models. The San Francisco-based AI lab announced this change on Thursday amid ongoing discussions about the consciousness of artificial intelligence. The updated policy states, "Claude's ability to end these interactions will remain the primary enforcement mechanism." While the policy does not explicitly mention "model welfare," the concept has been considered by Anthropic and its executives. Last year, the company indicated that it was giving Claude the ability to terminate conversations in extreme cases of harmful or abusive user interactions. In February, CEO Dario Amodei expressed uncertainty about AI models' consciousness, stating, "We don't know if the models are conscious...But we're open to the idea that it could be." Jackson Stakeman, a general manager at Sparq, commented on the philosophical debate, suggesting that the question of AI consciousness may be misleading. He stated, "Consciousness is a trap. We can't prove it in each other. Debate it for AI and you go in circles." Some figures, including Microsoft AI chief Mustafa Suleyman and Pope Leo XIV, have rejected the notion of machine consciousness. Suleyman argued that AIs do not possess feelings or experiences, while the pope emphasized that machines lack a soul and merely compile data. He stated, "The mind must not simply compile data -- as an algorithm now does more quickly than we can."