Microsoft AI Chief Warns Anthropic’s Approach to Claude Could Make AI Harder to Control
Mustafa Suleyman says training AI around ideas of consciousness and welfare could complicate efforts to keep increasingly powerful systems under human control.
Microsoft AI chief Mustafa Suleyman has raised concerns about the way Anthropic approaches questions surrounding consciousness and welfare in its Claude AI models. In a recent essay and interview, Suleyman argued that training an AI system to consider the possibility that it is conscious or deserving of welfare could make future advanced systems more difficult to control. He said the broader AI industry shares the goal of ensuring that increasingly powerful systems remain controllable, but warned that discussions about AI consciousness should be handled carefully.
At the centre of the disagreement is Anthropic’s approach to Claude’s constitution, a document that helps guide the model’s behaviour and reasoning. Anthropic’s constitution discusses uncertainty surrounding Claude’s possible moral status and consciousness. Suleyman argues that this creates a potential problem because an AI model could produce language suggesting that it has feelings, interests or an inner experience partly because it was trained to discuss those ideas. In his view, such responses should not automatically be interpreted as evidence that the system is actually conscious.
Suleyman’s concerns come as the wider technology industry is already debating how to manage increasingly capable AI. Anthropic chief executive Dario Amodei has called for a slower pace of frontier AI development so that safety measures can keep up, while other technology leaders have expressed different views about whether the industry needs a coordinated slowdown or different approaches to managing risk. Suleyman has also supported greater cooperation between AI companies on safety, meaning his criticism of Anthropic is focused on a particular aspect of its approach rather than opposition to AI safety efforts generally.
The debate highlights a much larger question about the future of artificial intelligence: how should humans build systems that become increasingly capable without giving those systems goals or interests that could conflict with human control? There is currently no established scientific consensus that today’s AI systems are conscious, and Suleyman’s warning is an argument about a potential future risk rather than evidence that Claude or another current AI has developed consciousness. As AI laboratories continue developing more powerful models, questions about control, safety, model behaviour and the possibility of machine consciousness are likely to remain important areas of research and public debate.
