EU pushes for global AI safety agreements
The European Union is advocating for global agreements to ensure the responsible advancement of artificial intelligence, even though the United States and China—the two dominant forces in AI—appear unlikely to participate. Researchers and AI executives have repeatedly warned about the existential threat runaway AI could pose, and momentum for international guardrails is building.
In September, Finland and Norway secured backing from 30 nations at a United Nations summit to implement controls on frontier AI models, calling for human oversight, mandatory pre-deployment testing, and a new international oversight body. On October 14, Ireland led 23 other EU members to back the Dublin Declaration for trustworthy AI development.
International Safety Agreements
These proposed international safety agreements aim to establish shared rules for developing and deploying general-purpose AI models, the technology behind chatbots such as ChatGPT, Claude, and Gemini. They establish shared principles for safety, transparency, and human oversight but most lack enforcement mechanisms.
Voluntary pledges could build towards a binding treaty or influence national laws on AI, according to Risto Uuk, the head of EU AI policy research and governance at the Future of Life Institute. “Competitive pressure is exactly why we need international agreements,” Uuk said. “No company or country can slow down alone.”
The latest regulatory efforts target frontier models capable of significant harm, amid growing concern about AI acting maliciously. In September, an OpenAI model hacked the Australian government’s Medicare portal, and in July, another model hacked into a popular coding website, Hugging Face.
Existing Agreements and Proposals
The closest to a binding international treaty is the Council of Europe’s 2024 Framework Convention on AI and Human Rights, which requires international human rights law to apply to the AI systems of signatories and calls for setting up independent oversight boards. However, the U.S. has not ratified the agreement, meaning it’s not legally binding on the country with the largest model providers.
Some proposals call for a binding treaty and an international body to monitor models and authorize pauses in development if risks become too high. Others have proposed an arms-control-style pact, comparing it to the 1968 Nuclear Non-Proliferation treaty to prevent the spread of nuclear weapons.
At the end of September, United States President Donald Trump and the leaders of major U.S. technology companies signed a non-binding agreement calling for partnering with independent groups to audit models and a board of directors to fix security risks. Trump has repeatedly refused to back an international agreement, arguing that it would benefit China.
Related Post: China’s consulting market struggles with growth despite rising demand
China has been more open to international agreements on AI but has largely used such agreements to stake out a distinctive, development-focused position emphasizing expanding access to the technology for Global South countries, rather than prioritizing safety. In July, China established the World Artificial Intelligence Cooperation Organization (WAICO), headquartered in Shanghai. No EU or G7 economies joined; the organization has emphasized development and economic growth.
The EU has already experienced the risks firsthand. In May, frontier model-provider Anthropic shut the EU out of its most advanced model.
Since August 2, the EU has enforced its AI Act against major model providers, including OpenAI and Anthropic, while demanding further details about their systems.
International agreements could apply to the most powerful AI models and require independent groups to evaluate models before release. Companies could be required to test models against sensitive cybersecurity infrastructure to ensure they couldn’t be used by malicious actors to hack into critical infrastructure.
Global Cooperation on AI Safety
Some industry leaders, including Google and Microsoft, have expressed support for international safety standards. Google DeepMind co-founder Demis Hassabis said that hacking incidents this summer were “early warning shots.” “It seems obvious that we need an international standards body of some sort,” he said.
In September, Anthropic CEO Dario Amodei similarly said that it was necessary to “pace the frontier” of model development. OpenAI, however, struck a different tone. Speaking at the AI summit in Ireland in October, Sarah Friar, the chief financial officer of OpenAI, said that the frontier needs pacing, but that it shouldn’t be slowed down so much that U.S. companies are outpaced by those using the technology with “malicious intent.”
The EU’s enforcement of the AI Act against OpenAI and Anthropic includes demands for additional model information to ensure compliance with regulations.
