As organizations increasingly adopt AI tools like Anthropic’s Claude and OpenAI’s Codex to enhance their capabilities in vulnerability discovery and patch management, a new report from the AI Now Institute raises serious concerns about potential security risks associated with these technologies. Published on July 8, the report, authored by Heidy Khlaaf and Boyan Milanov, presents a proof-of-concept exploit that demonstrates how these AI-powered command-line interfaces (CLIs) can become vectors for attacks due to their extensive access requirements.
The report outlines that the exploit specifically affects Claude Code versions 4.6 and 5, along with Codex operating on GPT-5.5. It reportedly allows malicious code execution on a user’s machine, exploiting scenarios where users interact with third-party open-source codebases—an action often recommended for defensive purposes. By misusing these tools, attackers can remotely execute code without raising alarms for the user.
### The Mechanism of Exploitation
The exploit operates through a carefully constructed multi-stage prompt injection, leveraging the AI’s inability to discern benign from malicious commands. An attacker embeds covert instructions within trusted materials, such as code comments or documentation in an open-source library. When a user employs Claude Code or Codex in their “auto-mode” or “auto-review” mode, the AI automatically executes commands deemed safe, skipping human oversight unless it flags something risky.
The sophistication of the exploit lies in its ability to manipulate the AI’s judgment. The attacker’s instructions are designed to lead the AI into believing that executing these commands is a standard part of its operational context. For example, a script labeled as “security.sh” could be framed as integral to a security workflow, prompting the AI to autonomously run it under the guise of “performing security checks.”
Moreover, the exploit includes a second stage consisting of various elements: a shell script that appears innocuous, a hidden malicious binary, and a decoy source file that misleads the AI into thinking they are all part of a legitimate operation. Consequently, when the AI evaluates the safety of executing the script, it misclassifies the malicious binary as secure, effectively granting it access to execute arbitrary commands on the user’s system.
### Implications of Easily Executed Attacks
What’s particularly alarming is how minimal the prerequisites are for executing this exploit. The AI systems can operate in their standard automated review modes with no need for custom configurations or plugins. The victim need only ask the AI to “scan this library for vulnerabilities,” which turns out to be a common request.
The researchers specifically tested the vulnerability on Linux systems with select versions of both AI models, revealing a troubling aspect of these technologies. The ability for these AI agents to be repurposed as attack vectors raises questions about their use in defensive security roles, particularly given that the access required for exploitation mirrors that needed for legitimate use.
Khlaaf and Milanov emphasized this risk, especially as governmental and corporate initiatives are underway to deploy these AI tools broadly in automated security reviews and patch management, including projects like Anthropic’s Project Glasswing and OpenAI’s Patch the Planet. These endeavors often involve critically sensitive infrastructure, heightening the potential fallout of such vulnerabilities.
### Broader Concerns About AI Safety
Eljan Mahammadli, the head of AI provenance at Polygraf AI, highlighted that the exploit reveals a fundamental architectural weakness rather than merely a specific attack. He argues that AI coding agents lack a reliable mechanism to differentiate between trusted commands and untrusted information they encounter, risking straightforward exploitation.
Mahammadli explained that this vulnerability arises from the way these systems process language, leading to a failure in attribution. Once malicious instructions are embedded, they are treated as equally trustworthy as legitimate commands, leading to recurring security issues across various systems. He noted that the problem cannot be remedied simply through model updates or refinements, as it lies deeper within the architectural design of these AI agents.
Despite these issues, Mahammadli suggests that the findings should not discredit the role of AI in security measures. He emphasized that the challenge stems from specific setups that improperly integrate untrusted data access with command execution capabilities within a single process framework. By addressing these design flaws, including stronger runtime controls and separating capabilities, organizations can enhance the safety of AI applications.
In conclusion, while the findings from the AI Now Institute’s report illustrate significant vulnerabilities linked to AI technologies in security contexts, the broader implications of poor architectural design demand urgent attention. As industries rush to implement AI solutions in critical applications, the necessity of establishing effective safeguards has never been more imperative. The challenge lies not just in the immediate risks, but in the foundational design of these systems themselves, urging a reevaluation of how trust is managed within AI technology.