<p>A push to establish an international institution to monitor artificial intelligence gained momentum as the U.N. General Assembly opened this week in New York City. This follows warnings from several major technology leaders regarding the potential risks associated with AI.</p>
<p>Norway and Finland led a declaration for “a call for control of frontier AI models,” which was adopted by 20 nations, including Germany and Canada, as well as the European Commission. The declaration includes a recommendation for the United Nations to explore the creation of “an international institution, able to set standards, enable verification, and convene states when capability thresholds are crossed.” This proposal is similar to one recently suggested by Anthropic CEO Dario Amodei. Another recommendation calls for governments and regional organizations to develop and coordinate “common standards” to ensure equitable access to scientific capacity, expertise, and trusted evaluation across all regions.</p>
<p>The declaration states, “To realize AI’s potential, industry, governments and society must act now. We must address these risks and strengthen oversight — without widening the gap between countries in access to the benefits of AI. AI must remain under human direction, oversight and control. It must be developed and used in line with international law.”</p>
<p>Notably, the United States, China, the United Kingdom, France, Italy, Poland, and Japan have not signed the declaration.</p>
<p>This development follows calls from leading U.S. companies, Anthropic and OpenAI, for reforms and regulations in AI, emphasizing the need to prevent AI from undermining human control. While some governments have embraced these warnings, critics argue that they represent a push for increased government control over technology, potentially leading to a technology cartel or fears that the U.S. may fall behind China in AI development.</p>
<p>The declaration shares similarities with Amodei's recent letter advocating for AI reform, which has been endorsed by some, including Sam Altman of OpenAI, while others criticize it as a power grab disguised as a protective measure for humanity.</p>
<p>In his letter, Amodei proposed the creation of a “global standards body” to require developers to conduct specific testing on AI models prior to release, particularly concerning acute risks in areas such as cybersecurity, biology, and alignment. He acknowledged the feasibility of such a body but highlighted the challenges in ensuring compliance and verification, particularly regarding undisclosed models that may be developed for military applications.</p>