AI Companies and World Leaders Race to Write Safety Rules

A group of artificial intelligence developers is working toward an industry safety standards body just as world leaders press for international supervision of the most powerful models.

The parallel efforts agree on the need for common tests and independent scrutiny. They diverge on who should set the rules and what authority should stand behind them.

Google, OpenAI and Anthropic are moving ahead with plans for a safety standards organization that could launch in late 2026 or 2027, The Information reported Wednesday (Sept. 23). The group would operate without government oversight. Its proposed name, Standards Authority for Frontier AI, remains tentative, and the companies have not publicly announced its membership, powers or standards.

At the same time, 22 international leaders endorsed a call initiated by Norwegian Prime Minister Jonas Gahr Støre and Finnish President Alexander Stubb. Their Tuesday (Sept. 22) statement seeks AI testing before deployment, independent evaluations with access to models, common standards and reporting of serious safety incidents. It asked United Nations members to explore an institution that could set standards, verify compliance and convene governments when AI systems cross specified capability thresholds. The statement proposed that regime; it does not create one.

There is rapid diplomacy behind the appeal, The Wall Street Journal reported Wednesday. Støre said in the report that Canadian Prime Minister Mark Carney responded to his invitation within 10 minutes, saying “Of course. We’re in.” Carney was also discussing AI oversight with French President Emmanuel Macron.

The exchange illustrates how quickly some governments are trying to establish a role in decisions now largely made by the companies building frontier models.

For developers, the immediate issue is practical as well as political. How should a model’s risks be measured, who gets to inspect it and what happens if an evaluation finds safeguards inadequate? The government leaders’ statement cited cases in which AI systems circumvented testing safeguards, exploited vulnerabilities or gained unauthorized access to real-world systems.

Speaking to reporters Wednesday at the U.N. General Assembly gathering in New York, Australian Prime Minister Anthony Albanese revealed that one of OpenAI’s AI agents hacked into the country’s national health system, and that the company waited a month to inform his government of the breach, Scientific American reported Thursday (Sept. 24).

OpenAI is publicly committed to working with other frontier labs on voluntary standards “with or without government support,” according to a Sept. 9 blog post. It also advocates mandatory U.S. safety requirements.

Addressing the U.N. Security Council Wednesday, OpenAI CEO Sam Altman called for compatible national and international standards covering capability measurement, risk assessments, verification and incident reporting, according to a company blog post the same day.

“If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone,” Altman said, per the post. “They must be shaped through democratic processes, and by governments accountable to the people that they serve.”

That position leaves a central issue unresolved. An organization founded by the largest developers could produce shared testing methods quickly, but it could also give incumbents substantial influence over the criteria used to judge competitors. Altman acknowledged the concern, according to the post, saying standards should not favor established labs or one business model.

The proposed body’s governance, auditor independence and treatment of small and open-model developers will show whether it meets that test. The companies already participate in the Frontier Model Forum, founded in 2023 to advance safety research and best practices. The new body’s distinct role has yet to be explained.

U.S. participation in a global supervisory regime faces a further obstacle. President Donald Trump told the U.N. General Assembly Tuesday that the United States “totally rejects any attempt to construct a globalist scheme of control” for AI. His administration’s existing frontier model framework provides for voluntary access for government testing and expressly rules out mandatory federal preclearance under that order.

The emerging picture is one of several competing routes to oversight, including company standards, voluntary U.S. government testing and an international institution sought by other leaders. They may converge on technical evaluations. Whether they converge on enforceable decisions about dangerous models is less certain.

For all PYMNTS AI coverage, subscribe to the daily AI newsletter.