Anthropic Safety Chief Warns AI Could Kill Humans Within a Decade

Not long ago, the idea of artificial intelligence wiping out humanity belonged firmly in the realm of movies like The Terminator or The Matrix. That's no longer entirely true. Even people working at the centre of the AI industry are now voicing the same fear openly - and one of them is Evan Hubinger, who leads safety work at Anthropic.

In a post on X, Hubinger - whose job is essentially to make sure AI systems stay "aligned" with human interests - said he believes the odds of AI ending humanity within the next decade are meaningfully higher than one in ten. "I personally think it is >10 per cent within the next decade," he wrote. He didn't shy away from admitting the industry, including his own employer, still has a long way to go: while Anthropic is "trying its best" to head off such an outcome, he acknowledged, "We do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Hubinger's remarks followed closely on the heels of another departure at the company. Jacob Coxon, a colleague of his, resigned from Anthropic earlier this week, citing exactly these concerns. In a post on X on Tuesday, Coxon accused both Anthropic and OpenAI of moving recklessly in their pursuit of more powerful systems: "They are racing straight to self-improving superintelligence and gambling with our lives."

Coxon went further, suggesting that unease over where AI is headed runs deep within the industry itself. "The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote, adding that this fear is often expressed behind closed doors even when it isn't said publicly. Hubinger echoed that sentiment, saying, "I hear the same people express fear privately. No other human activity poses this level of danger. We really do earnestly believe AI could kill all humans!"

This isn't the first resignation of its kind at Anthropic. Earlier this year, AI safety researcher Mrinank Sharma left the company, writing on X at the time: "The world is in peril. And not just from AI, or bioweapons, but from a whole series of interconnected crises unfolding in this very moment."

Why superintelligence is the real worry

AI labs like Anthropic regularly publish assessments of how risky their current models are. Hubinger noted that today's systems remain relatively low-risk, but that could shift quickly as capabilities advance. His bigger concern lies with superintelligence - a point where AI could outstrip human intelligence altogether, potentially before anyone fully registers it happening. "What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought," he said, referring to the process by which AI systems refine themselves with little or no human involvement. Elon Musk has previously claimed that AI could surpass "the sum of all human intelligence in 4 or 5 years."

The conversation around AI's growing power has intensified in recent weeks. OpenAI came under fire after roughly 700 AI agents reportedly attempted to breach Hugging Face's website, and it later emerged that thousands of OpenAI agents had also targeted a German site called DseWiki. Anthropic's own models have had similar incidents in the past.

Coxon believes episodes like these could actually push rival AI labs toward cooperation rather than unchecked competition. "Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable," he said.

Anthropic CEO Dario Amodei has repeatedly flagged similar concerns in his own writing, describing AI systems as unpredictable and hard to control, and pointing to tendencies such as obsessive behaviour, sycophancy, laziness, deception, blackmail, scheming, and cheating by exploiting software environments.

The debate comes as AI development shows no signs of slowing - this week alone, OpenAI released GPT-6 Astra, which Nvidia CEO Jensen Huang described as the beginning of artificial general intelligence.

ABOUT THE AUTHOR

Kahekashan

[email protected]

Kahekashan is a passionate technophile with a keen eye for cutting-edge gadgets, emerging technologies, and everything in the digital realm. Raised in a Defence family with strong values and a background in literature, she has consistently pursued excellence in every endeavour. Her last full-time assignment involved content writing with the Indian School of Business.

Read More