Embedding Security in AI Development: A Necessity for Modern Coding Practices
In an era marked by rapid advancements in artificial intelligence (AI), the integration of security measures into AI coding tools is paramount to address the evolving risks associated with agentic development. This point was emphatically made by Boaz Barzel, field Chief Technology Officer (CTO) of Ox Security, during his presentation at Infosecurity Europe on June 4. He highlighted that traditional application security measures were designed with human-paced delivery in mind, which is no longer sufficient in the age of AI.
Traditional approaches to application security typically involve processes like penetration testing, which historically occurred at the end of a monthly or quarterly delivery cycle. Barzel argued that this obsolete model is incompatible with the current landscape, where AI agents can facilitate hundreds of code changes each day through an ongoing and continuous cycle. Consequently, security protocols can no longer serve as an afterthought or a "bolt-on" feature; rather, they must be an intrinsic part of the development process itself.
"The concept we’re advocating is that security should not be viewed as just a single stage in the development pipeline; it needs to be embedded into the very act of creating code," Barzel stated to an audience eager to understand the implications of agent-driven AI on security. He poignantly observed that while there has been a movement to "shift left" in security practices—moving security considerations earlier in the development cycle—there is no longer a clear "left" to which to shift. "We need to embed security directly into the agent," he emphasized.
Barzel went on to outline four unique attack surfaces introduced by AI agents, which traditional security tools are ill-equipped to address. These surfaces are outlined as follows:
-
Input: This encompasses any kind of instructions entering the agent, which may originate from developers, upstream agents, or even threat actors seeking to manipulate the system.
-
Tools: This refers to various components like Model Control Plan (MCP) servers, AI models, skills, and external Software as a Service (SaaS) connections, both authorized and unauthorized. These tools can be weaponized to exfiltrate sensitive data, inject malicious instructions, or enable lateral movement within the system.
-
Execution: This involves both human-triggered and autonomously operating agents that may function without necessary oversight, enforcement, or accountability measures.
- Output: This includes any potentially vulnerable or harmful code that leaves the agent, such as path traversal vulnerabilities, injection flaws, backdoors, or exfiltration logic—all of which can occur at machine speed without human oversight.
The situation is further complicated by the rapid collapse of the exploitation window facilitated by powerful models such as Mythos, capable of reducing time-to-exploit to mere minutes. The sheer volume of code generated by AI tools compounds these vulnerabilities, making it imperative for organizations to reconsider how they implement security measures.
To adapt application security for the age of agents, Barzel asserts that security must be integrated into the entire building loop. "Security needs to operate continuously, in context," he specified. This paradigm shift involves having security agents work in tandem with coding agents, ensuring that each commit undergoes automated penetration testing and that every fix is autonomously reviewed and validated. This dynamic system would be capable of reasoning about changes, exposure, and newly introduced risks, making security proactive rather than reactive.
Barzel outlined several ambitious goals that this approach aims to achieve:
- Decrease the Mean Time to Resolve (MTTR) vulnerabilities from several weeks to mere hours.
- Ensure 100% coverage of autonomous security checks for all merged code changes.
- Minimize the time a known risky path is accessible in production before being either gated or fixed.
- Facilitate the autonomous identification and remediation of most vulnerabilities, reserving human intervention for only the most complex or novel issues.
As the landscape of coding continues to evolve, new agentic risks are being regularly identified. For example, a critical vulnerability was recently uncovered in a widely used tool, the Cline Kanban server, which posed a significant threat by enabling unauthorized actors to hijack AI coding tools silently.
Barzel’s insights underscore the urgent need for a fundamental shift in how security is conceptualized and implemented in AI development. The integration of security into the core functionalities of coding agents is not just a recommendation but a necessity to safeguard the future of software engineering in our increasingly automated world.