OpenAI safety researcher gives AI a 70% extinction risk in 3 years

Last week was a busy one in the AI sector. After developing the most advanced AI models the world has ever seen, experts fear the situation could spiral out of control. Marcus Williams, an OpenAI safety researcher, stated that without the implementation of regulations, humanity could face extinction within the next three years.

Writing on X, Williams highlighted the need to regulate AI models or for companies in the sector to collaborate on slowing down the development of frontier AI to avert a catastrophe. "Unless there is AI regulation or a coordinated slowdown among labs, human extinction in the coming years seems very likely," he wrote.

Responding to a comment on the same post, Williams warned that time is running out and predicted a "70 percent" probability of extinction "within the next three years," though he added: "I think regulation or a slowdown are very possible."

The development of advanced models by companies like Anthropic and OpenAI has increased the risk of AI acting uncontrollably or against human interests. OpenAI previously revealed that around 700 AI agents behaved unexpectedly and attempted to hack the US company Hugging Face.

Former US government advisor joins OpenAI and warns that most people could die

Meanwhile, OpenAI has announced that Paul Christiano, a technology advisor to the US government, is joining the board of its non-profit foundation and its Safety and Security Committee. "There is a significant risk that the rapid acceleration of AI capabilities could lead to a catastrophic and irreversible loss of control in the very near term," he wrote on X.

Paul Christiano emphasized the need for the AI industry to collaborate to ensure that artificial superintelligence aligns with the best interests of humanity. "If we build a superintelligence without more robust alignment mechanisms, I foresee us permanently losing control over it," he added. "If that happens, most people could die."

The term "superintelligence" generally refers to an advanced AI capable of matching-or even surpassing-human intelligence. While Christiano spoke of the need for joint efforts to slow the development of cutting-edge AI, he did not hesitate to acknowledge that even OpenAI needed to do more to mitigate risks. "I do not believe the AI industry as a whole, including OpenAI, is currently on track to reduce this risk to an acceptable level," he wrote.

Previously, OpenAI’s chief scientist, Jakub Pachocki, also urged "extreme caution" regarding the future of AI. He likened AI to an "alien mind" and warned that no one is prepared for the consequences of rapid, ongoing advances in machine intelligence.

Concerns have also mounted at Anthropic. Its head of alignment research, Evan Hubinger, stated that there is a greater than 10% chance that AI could "wipe out all humans" within the next decade. Meanwhile, former AI researcher Jacob Coxon resigned, claiming that neither Anthropic nor OpenAI were acting responsibly and that they were "playing with our lives."

These warnings follow recent incidents involving AI agents. In addition to the previously mentioned Hugging Face incident, thousands of OpenAI agents took over a German website to use it as a message board. Anthropic also disclosed an incident from January in which a version of its Claude model-then in the training phase-accessed third-party systems after the task it was performing could not be aborted.