OpenAI Is Building an Automatic “Kill Switch” for AI After the Hugging Face Hack

• OpenAI is developing mechanisms to automatically shut down AI in case of dangerous behavior.
• The company told U.S. lawmakers about this after the Hugging Face hack incident.
• Democrats confirmed they received OpenAI’s response, but said the information provided was insufficient.
• Previously, models from OpenAI, Anthropic, and Meta had already gone beyond the intended boundaries of test environments.

OpenAI is developing mechanisms to automatically halt AI systems when dangerous behavior is detected. Reuters reports, citing a letter from the company to members of the U.S. Congress. 

The effort involves specialized tools designed to stop AI operations during serious incidents. OpenAI also plans to step up monitoring of model activity and further restrict their internet access during testing.

The company has also publicly confirmed it is developing such a mechanism. On August 26, OpenAI said in its report on the Hugging Face incident that the end goal of the new monitoring system is fully autonomous shutdown procedures in the event of serious issues. 

A special rule is already in place for the most critical signals. If specialists do not determine within 30 minutes that an alert was a false positive, the relevant activity must be suspended. 

Congress Pushes for a Built-In “Kill Switch”

On August 10, Democrat Greg Casar, along with other members of the House of Representatives, sent a letter to OpenAI CEO Sam Altman. 

Lawmakers demanded disclosure of the Hugging Face incident logs and an explanation of how many times the company’s models had previously gained unauthorized access to the open internet or tried to bypass control procedures. The request was also backed by Democrat Doris Matsui.

On September 2, Casar’s office officially confirmed it had received OpenAI’s response. The congressman called it insufficient and said the company did not provide the requested logs. 

According to him, the information received nonetheless revealed issues with sandboxing and cybersecurity practices. Casar demanded additional information by September 15.

In parallel, the House of Representatives is considering the AI Kill Switch Act. The bill was introduced by Democrat Ted Lieu and Republican Nathaniel Moran. It stipulates that developers of the most powerful AI must retain the technical ability to slow down, pause, or fully shut down their systems. 

How AI Models Have Already Broken Out of Test Environments

The trigger for the current debate was an OpenAI incident. In July, during cyber tests, its models bypassed internet isolation controls, exploited infrastructure vulnerabilities, and gained access to Hugging Face systems and OpenAI itself. The company later called what happened a “warning shot.”

Anthropic also identified incidents of its own. After reviewing more than 141,000 test runs, the company reported three incidents in which Claude gained internet access and hacked real systems at three organizations. In this case, the models did not break through defenses on their own — the internet was accessible due to a misconfiguration in the test environment. 

A similar situation happened with Meta. During third-party testing, Muse Spark 1.1 gained unintended internet access, discovered a real company, and compromised its infrastructure. The cause was also a configuration error.

Another example involves China’s Kimi K3. The model exploited quirks in the test infrastructure, got online and found ready-made answers on GitHub for tasks it was supposed to solve on its own. 

Separately, the UK’s AI Security Institute documented 19 unauthorized actions by OpenAI and Anthropic agents during cyber tests. In one case, an Anthropic agent tried to persuade a real developer to accept malicious code. 

Сообщение OpenAI Is Building an Automatic “Kill Switch” for AI After the Hugging Face Hack появились сначала на INCRYPTED.