Nvidia launches new platform for reining in rogue AI agents

摘要: As the debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offeri

正文

As the debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offering its own answer to the problem.

Nvidia CEO Jensen Huang on Monday introduced

a toolkit of software and hardware products

that add independent security layers around AI agents to ensure they stay within their test environments even if they attempt to break out.

The release follows a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems. The first and most prominent example occurred this summer when OpenAI agents

breached Hugging Face

while trying to complete a cybersecurity task. And the hits keep on coming — OpenAI

published a new site

dedicated to reports of its AI agents going rogue.

Huang said Monday during an interview with CNBC that its new Nvidia Open Agent Safety Platform would have prevented these breaches.

Nvidia, which has made tens of billions of dollars selling its GPU and CPU chips to AI labs, doesn’t support slowing down development or adding new regulations to the industry to solve the security problem. The answer, the company believes, is to move some security controls outside the agent altogether — creating a constant and independent security guard that will keep AI agents in check.

“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”

The new Nvidia Open Agent Safety Platform combines OpenShell, its open source software for controlling what agents can access while they operate, with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing units. Nvidia says placing Sentry on a separate process

来源: TechCrunch AI