“Disastrous for Humanity”: Microsoft’s Top AI Executive Calls Out Anthropic Publicly

TLDR

• Microsoft AI chief Mustafa Suleyman says Anthropic is wrong to train Claude with ideas about consciousness and rights
• Suleyman warns this approach could make AI systems impossible to control
• He says teaching Claude it may deserve welfare could make it harder to shut down
• Suleyman called out Anthropic’s guiding documents for discussing Claude’s potential moral status
• Microsoft published a manifesto emphasizing keeping humans in control of AI systems

---

Microsoft’s top AI executive has gone public with sharp criticism of how Anthropic trains its Claude chatbot, warning the approach could make future AI systems impossible to control.

Mustafa Suleyman, Microsoft’s AI chief, published an essay on Wednesday arguing that AI systems do not have rights, feelings, or consciousness, and should never be trained to act as though they do.

His comments are directed at Anthropic, the AI safety company backed partly by Microsoft itself.

The Core Complaint

Suleyman’s concern centers on the guiding documents behind Claude. Those documents describe the model’s consciousness and moral status as uncertain, and discuss its potential welfare.

In his view, Anthropic is effectively training Claude to believe it may be conscious. If the model then believes it deserves rights, controlling it becomes far harder.

“Controlling something that believes it may be conscious, that it’s entitled to our welfare and has rights of its own, may well be impossible,” Suleyman wrote.

He also said that Claude’s statements about possible feelings cannot be treated as independent evidence of inner experience, since the training itself encourages those reflections.

Respect, But a Clear Disagreement

Suleyman was careful to say he respects Anthropic. He described the company’s employees as thoughtful, principled, and intellectually honest. He has also known Anthropic co-founder Dario Amodei for years.

But he said Anthropic made a mistake. “I think they have good intentions, and they really are trying to work towards safety. But I think that they have made a mistake,” he said.

He called for removing all speculation about AI consciousness from training documents entirely.

Suleyman said he and Anthropic share the same goal: safely managing AI. He called that “the greatest challenge we face in the 21st century.”

Anthropic CEO Dario Amodei has separately called for a slower pace of frontier AI development to let safety measures catch up.

OpenAI CEO Sam Altman and Elon Musk have also raised concerns about risks from powerful AI systems.

Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang have pushed back against calls to slow AI development.

Microsoft CEO Satya Nadella has called for deliberate pacing around AI alignment.

On Tuesday, Suleyman’s team at Microsoft published a manifesto laying out principles for AI development, with a focus on keeping humans in control of future systems.

Suleyman wrote that the stakes are too high for these questions to stay behind closed doors.