Karmactive Team
An AI researcher set off a social media frenzy by quitting his job at Anthropic and accusing both OpenAI and Anthropic of acting irresponsibly, igniting concern about the rapid pace of AI development. Jacob Coxon’s post on X, viewed more than 70 million times, said the companies are “gambling with our lives.”
Coxon, who has worked as a researcher at both companies, said in his post that the people building AI “earnestly believe that it could kill us all by the end of the decade.” “Do not underestimate the power of this technology,” Coxon wrote. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
Anthropic alignment lead Evan Hubinger echoed the warning in a post on X late Tuesday. “Jacob is correct here. We really do earnestly believe AI could kill all humans! I personally think it is greater than 10% within the next decade,” Hubinger wrote. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
OpenAI chief scientist Jakub Pachocki published a blog post on Sunday warning that no AI company has “solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.” “I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established,” Pachocki wrote.
The warnings come as Anthropic and OpenAI barrel toward potentially historic initial public offerings while releasing increasingly advanced models. In 2023, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei signed a statement saying “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Some members of Congress have taken steps to address AI’s rapid advancement. Reps Jay Obernolte and Lori Trahan introduced the FRONTIER Act to establish a framework for governing advanced AI models. Sen. Bernie Sanders and Rep. Greg Casar introduced the Ban Artificial Superintelligence Act, which would temporarily pause advanced AI development until the federal government establishes safety rules.
“Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway,” Trahan wrote on X on Wednesday. “It’s past time for Congress to get off the sidelines and do its job.”
