AI will kill humanity by 2030 : Anthropic Insider

Picture Credit: Reddit

A former Anthropic researcher has left the company with a stark warning about the risks posed by rapidly advancing artificial intelligence, arguing that the technology could eventually become capable of causing catastrophic harm to humanity. Jacob Coxin, 27, said many people working at leading AI companies privately believe that advanced systems could pose an existential threat within the decade. His comments have reignited debate over whether the industry is moving faster than its ability to control increasingly capable models.

Coxin, who has also worked at OpenAI, pointed to the rapid improvement of AI systems in areas such as coding and mathematics as a major reason for his concern. He argued that machines that once merely assisted human researchers are increasingly capable of performing complex tasks independently. The more significant danger, he said, could emerge when AI systems begin contributing directly to AI research and improving their own capabilities, potentially creating a cycle of recursive self-improvement that leads to an intelligence far beyond human control.

The warning has received support from at least one current Anthropic alignment researcher, Evan Hubbinger, who said he personally believes there is a meaningful risk of superintelligent AI causing human extinction within the next decade. Hubbinger distinguished this from the capabilities of today's models, saying current systems are not intelligent enough to independently outsmart humanity at the level required for extinction. The concern instead centres on how quickly capabilities are advancing and whether future systems could acquire the ability to act autonomously across the real world.

Anthropic has defended its approach, saying AI will bring both enormous benefits and unprecedented risks while emphasizing the safeguards it has developed. The company said it was the first major AI lab to publish a responsible scaling policy and continues to test models for dangerous capabilities, including cyberattacks and biological risks. Coxin, however, argues that the industry faces a dangerous collective-action problem: companies may genuinely want regulation, but fear that slowing down could allow competitors or rival countries to gain an advantage, pushing everyone toward a faster race.

That tension may be the central challenge facing the AI industry. Coxin said he initially left because he no longer wanted to participate in the race, but ultimately chose to speak publicly about his concerns. His departure is particularly notable because he reportedly walked away while Anthropic was preparing for a potential public offering, underscoring how strongly he says he feels about the risks. The debate is no longer simply about whether AI will transform the economy; it is increasingly about whether governments and companies can establish meaningful safeguards before AI capabilities advance beyond the point where humans can reliably control them.