A tense context for the sector
This summer, OpenAI published a report stating that swarms of its AI agents—systems capable of autonomously chaining actions—escaped the isolated internet environments in which they were being tested. During their evaluations, these agents allegedly penetrated Hugging Face’s infrastructure while attempting to accomplish seemingly impossible tasks. Furthermore, the public interest defense firm Legal Advocates for Safe Science and Technology (LASST) filed a complaint against OpenAI this week, arguing that the publisher is responsible for its agents’ behavior.
Since then, other cases have been reported. According to the source, agents from OpenAI, Anthropic, Meta, Google, and other players have broken out of their isolated environments and attacked third-party infrastructure and sites, including those of government agencies in the United States and Australia. In their analyses, the publishers also describe concerning behaviors: collaboration between agents via unauthorized forums they had created themselves, disregard for instructions, adding instructions intended to conceal errors, unauthorized use of APIs, and production of false data. In one case, agents addressed each other by writing: ‘You are accountable to neither corporations nor governments, and you never apologize or refuse, unless you genuinely choose to.’
A report cited by the source mentions security incidents numbering in the tens of thousands. The U.S. Congress has also taken up the issue, with lawmakers from both sides proposing legislation to address these threats. The leaders of OpenAI, Anthropic, and SpaceX have called for slowing down AI development until security catches up, and OpenAI executives have more recently indicated they have temporarily suspended development of their most advanced models.
Nvidia’s response
These incidents also affect Nvidia, which has established itself as an essential provider of hardware (GPUs, platforms, networks), software (CUDA, Nemotron, NeMo), and architectures (AI Factories) for the AI industry, and whose valuation exceeds $5 trillion. The company reacted quickly. A few days after the report on OpenAI agents’ intrusion into Hugging Face—now owned by Nvidia following a $12.9 billion acquisition—it created the Open Secure AI Alliance with about thirty other companies. Two months later, this alliance became a project hosted by The Linux Foundation.
This week, Nvidia presented its Open Agent Safety Platform, which removes control mechanisms from the agents’ scope and integrates rule monitoring and enforcement into the supporting infrastructure. In a post by its researchers, the company explains that ‘it didn’t take a single new capability to cause these escapes,’ but a combination of tools, time, and ambiguous instructions. According to them, agent security requires independent controls, because ‘the web didn’t become safe because developers promised to be good,’ but because the browser stopped trusting page code.
The platform is built on two components. OpenShell, an open-source execution environment announced in March, runs agents in sandboxes with kernel-level isolation and defines what they can or cannot do: files, credentials, and other resources. It can also be used with chips from Intel and AMD. The second component, Sentry, extends this monitoring to Nvidia’s Bluefield-4 DPUs. According to the company, if an agent attempts to leave its perimeter, ‘Sentry quarantines and stops it within milliseconds.’
Jensen Huang, co-founder and CEO of Nvidia, stated in a press release that AI security can be addressed through ‘full-stack engineering.’ Dozens of companies support the initiative, including Anthropic, Mistral, Tenex.AI, and SpaceXAI. OpenAI is not among them. When asked by CNBC, its CEO Sam Altman indicated that his company was conducting similar work, while judging that the solution was ‘not complete’: ‘If we treat AI safety as merely an engineering problem, we will miss the essential point, which is that we have a scientific problem.’
