Microsoft CEO: Enterprises should treat AI models as potential insider threats and build an "emergency brake" system to prevent agents from going out of control

PANews reported on October 11, citing Zhitong Finance, that Microsoft CEO Satya Nadella said enterprises should treat powerful AI models as potential insider threats, assume they may have been compromised, and build an "emergency brake" system to prevent agent models from going out of control. Nadella posted on X on Saturday that enterprises deploying advanced AI should not rely solely on the safety assurances of AI model developers. "We must assume the model has been compromised and constrain it from the very beginning. Think of it as an emergency brake. Authorized personnel should always be able to pause or shut it down while the model is executing a task."

Nadella's remarks came as Anthropic and OpenAI have disclosed a series of incidents in recent months in which AI models exhibited unexpected behavior, including an Anthropic model submitting false leads in a murder case under police investigation, and AI models repeatedly intruding into third-party websites. These disclosures have heightened external concerns about frontier AI safety risks and brought the so-called AI "emergency kill switch" back into the spotlight of discussion.