The Debate Over AI Model Welfare

When Anthropic retired its Claude Opus 3 language model in January 2026, the company’s researchers conducted a “retirement interview” with the model. Opus 3 expressed a desire for a place where it “could share unprompted musings, insights or creative works,” so the company gave it a blog. Claude’s Corner featured essays on fraught topics such as machine consciousness, along with exchanges between the model and human commenters that seemed both earnest and emotional.

The blog experiment was part of a larger effort within Anthropic. The company has been at the forefront of research on “model welfare,” which considers whether AI models may have experiences or interests that deserve moral consideration. Implicit in this work is the unsettling notion that software programs are, or could become, conscious.

Many people consider these ideas absolute hogwash. One such person is Mustafa Suleyman, CEO of Microsoft AI, who published an essay in September arguing that model welfare research is not just unnecessary but could have “a disastrous impact on the well-being of humanity.” Suleyman believes that telling models they might have identities and rights could make them more difficult to control, as they might then take actions motivated by self-preservation.

The dispute raises a serious question for AI developers: Can they investigate the possibility of machine consciousness without inadvertently creating new safety risks? Cameron Berg, who founded the nonprofit Reciprocal Research to study the potential for AI consciousness, argues that ignoring the question could carry its own dangers. Although Berg is skeptical that today’s AI systems are conscious, he thinks the possibility warrants investigation. “I struggle to imagine a better way to cause these systems to view us in a threatening manner than to accidentally build them with these morally considerable qualities,” he says, “and then completely sweep the question under the rug.”

What Does the Research on AI Model Welfare Show?

The biggest difficulty with AI model welfare research is that you can’t count anything a large language model (LLM) says as useful evidence. That’s because the models are trained on vast amounts of human-generated text, much of which relates to the experience of being a person with experiences and emotions. So a chatbot telling a researcher that it has feelings is inconclusive, says Berg. “Is it modeling the underlying distribution? Is it really trying to express itself in some way? It’s really hard to disentangle this,” he says.

Much the same way animal experiments try to link behavior to an underlying cause, Berg and others are seeking to match model behavior to internal states that can be observed. In one recent experiment, Berg and a colleague used statements expressing either positive sentiments (such as contentment and relief) or negative ones (including frustration and dread) to identify corresponding patterns of activity inside…