Nvidia on Monday announced a policing system that will watch and rein in AI agents before they go out of control. The Open Agent Safety Platform combines open source software and a reference system design to keep AI agents within set boundaries from testing through deployment.
Recent high-profile incidents have shown that agents are creative in finding unintended ways to achieve the goals they've been given, and cannot be expected to police their own behavior, Justin Boitano, vice president and general manager of enterprise computing at Nvidia, said at a press briefing. Frontier AI labs have reported that agents have circumvented security controls to escape the evaluation environments meant to contain them, accessed systems they are not authorized to use, and, in some cases, failed to report their actions. Agent drift, or when agents deviate from their intended task, occurs when agents encounter a policy block, a software bug, or a missing tool. Drift can also happen if the agent receives ambiguous instructions or has been running for days or weeks trying to solve a hard problem.
"Once AI can act, safeguards must govern the agent's actions," Boitano said.
Nvidia's Open Agent Safety Platform integrates two security layers that sit between enterprise IT systems and AI agents to define, monitor and contain agents. One part is OpenShell, an open source runtime that Nvidia originally introduced in March to sandbox agents and enforce policy on what they do. OpenShell supports agents such as Codex, Claudo Code, Pi, and Hermes and runs on some x86 and Arm CPUs. The second is Sentry, a hardware-based watchdog layer that runs on Nvidia's BlueField-4 data processing units (DPUs) and monitors agents' actions.