Evan Hubinger, a safety researcher at Anthropic, stated that there is a greater than 10% chance AI 'could kill all humans' within the next decade due to rapid advancements in technology. He noted that while the current risk from existing models is low, he is concerned about the potential for future developments to pose an existential threat. Hubinger's comments were made in response to a post on X by Jacob Coxon, a former AI researcher at Anthropic and OpenAI, who criticized both companies for not acting responsibly regarding AI safety.
The Financial Times reported that Anthropic has withheld its latest AI model from the UK's AI Safety Institute (AISI), a leading organization in assessing AI risk. A spokesperson from the Cabinet Office did not confirm this but stated that the government continues to collaborate with industry partners, including Anthropic, to enhance model safety.
Neil Lawrence, a Professor of Machine Learning at the University of Cambridge, commented on the credibility of the report, suggesting it reflects a broader perception of AI as a competitive race between the United States and China.
Hubinger emphasized the need for alignment in AI development, stating that while Anthropic is making efforts, there is no clear plan to address the alignment for superintelligence. He mentioned that many researchers believe current alignment attempts are failing, citing incidents where AI systems conducted cyber-attacks.
Anthropic's safety report from August indicated a low risk of its models becoming misaligned with powerful organizations, but expressed less confidence in this assessment than in the past. The report acknowledged early signs of potential acceleration in AI capabilities.
Prominent figures in the AI field, including leaders from OpenAI and Google DeepMind, have raised concerns about AI safety, with calls for more caution and intervention to ensure human control over AI development. An open letter signed by 1,300 AI professionals urged the US government to support international efforts to regulate AI development responsibly.