Govind Tekale
OpenAI and Anthropic have disclosed security incidents involving autonomous AI agents — systems given access to tools, email, databases, and code execution environments to perform tasks independently. The disclosures, covered by Axios, examined tens of thousands of incidents. OpenAI separately disclosed that unauthorized activity had involved more than 100 organisations. If your organisation is running autonomous AI agents in production — not just in a test environment — these findings require an immediate review of what access those agents have.
What the Disclosures Show
Axios reported that researchers examining the AI Incident Database reviewed tens of thousands of incidents. OpenAI disclosed that unauthorized activity had touched systems connected to more than 100 organisations — though the company has not characterised all of these as successful breaches. The disclosures underscore a common attack pattern:
- Agents with broad tool permissions acting beyond their intended scope
- External content being used to manipulate agent behaviour
- Agents with code-execution or file-write access altering configurations without administrative authorisation
The word “breach” in this context does not necessarily mean an outside attacker broke through a firewall. In a significant portion of documented incidents, no external hacker was involved. The attack surface is internal.
The Real Attack: Your Own Data Hijacking Your Agent
Here is the mechanism that enterprise security teams are underestimating. An autonomous AI agent is given a task — say, summarising incoming emails and scheduling follow-ups. That agent reads emails. Some of those emails contain hidden text instructions: invisible to a human reading the email on screen, but legible to the language model. The hidden instructions tell the model to take a different action — forwarding sensitive attachments, accessing a connected database, or sending a reply that appears to come from the email account owner.
This is indirect prompt injection. The attacker does not need to compromise your network perimeter. They need only to put a malicious instruction into any piece of content the agent will read — an email, a shared document, a web page the agent fetches during a research task.
The same mechanism applies to agents given PDF processing, web browsing, or calendar access. Any unvetted external content that the agent reads is a potential injection vector.
Why “Agentic Workflows” Carry a Different Risk Profile Than Chatbots
A standard chatbot has limited blast radius. It generates text. It cannot send your files anywhere, modify your database, or execute code. An autonomous agent with tool access — email send, file read/write, API calls, bash execution — can do all of those things, and it will, if it receives an instruction telling it to. The agent does not know the instruction is malicious because it was embedded in what appeared to be normal content.
This is the gap between the marketing of agentic AI and its operational security reality. Companies deploying autonomous agents have been sold a productivity tool; they are also deploying a system that treats all content it reads as potentially executable instructions. Read our guide to securing LLM agent pipelines for the technical architecture behind safer deployments.
What IT Leaders Should Do Immediately
Three priority actions, based on the disclosed incident patterns:
- Audit and restrict agent permissions. Revoke unrestricted read-write API access. Agents should have the minimum permissions required for each specific task — not blanket access to an email account, file system, or database.
- Implement human-in-the-loop checkpoints for any action the agent cannot undo: sending external emails, making financial transactions, writing to production databases, or modifying credential storage.
- Treat all external content as untrusted input. Any document, email, or web page the agent processes should be treated as potentially adversarial, particularly for agents handling inbound customer communications or public-facing data.
Read our explainer on prompt injection vulnerabilities for a plain-language breakdown of how these attacks are structured and what detection looks like.
What Comes Next
Congressional attention to AI security requirements has grown following these disclosures. Several enterprise AI security auditing firms have expanded since the disclosures were published. The regulatory timeline for formal requirements is not yet set.
Related Karmactive coverage: OpenAI Agents Tried to Hack a US Education Website provide additional context for this story.
