CyberSecurity SEE

AI-Written Patches Evolve into Attack Maps for Hackers

AI-Written Patches Evolve into Attack Maps for Hackers

This summer, the maintainers of vm2, a widely used open-source JavaScript sandboxing library that boasts an impressive 1 million downloads weekly, faced a major challenge as they addressed 34 separate vulnerabilities relating to sandbox escape. Remarkably, Anthropic’s Claude—a cutting-edge AI model—co-wrote 19 of these fixes. Nevertheless, the situation took a troubling turn when a security researcher directed an AI agent to investigate one of the advisories for weaknesses, and astonishingly, the agent successfully pinpointed a method to bypass the patch.

Shayaun Nejad, the lead product security engineer at Rubrik, conducted a thorough vulnerability variant analysis on the vm2 patch while preparing for a presentation at the recent [un]prompted AI security conference held in San Francisco. His aim was to shed light on the multitude of challenges that developers currently grapple with, particularly given the influx of vulnerabilities discovered with the aid of AI. He shared his observation that even after fixing certain issues, these same bugs appeared once again during new rounds of testing despite the implementation of AI-assisted pipelines.

Nejad elaborated on this phenomenon, pointing out, “Sometimes the [vulnerability] variant wouldn’t be caught until the model variation got better or the model used chaining analysis at a higher degree once it improved in order to find those variants.” He emphasized that vulnerability variant analysis is increasingly becoming a focal point, thus prompting organizations to adapt.

The rapid increase in published Common Vulnerabilities and Exposures (CVEs) can be attributed to AI assistance, rendering software developers more transparent about the various vulnerabilities their platforms may possess. Moreover, the patches intended to resolve these vulnerabilities may inadvertently serve as guidelines for potential attackers. Nejad lamented that it can be as straightforward as deploying an AI agent on a codebase containing historical vulnerabilities and instructing it to search for variants.

Rubrik’s experience serves as an indicator of a broader trend noted by researchers: AI models have demonstrated impressive offensive capabilities—effectively identifying vulnerabilities and devising exploits—while their defensive attributes, such as generating secure code and adequate patches, have not kept pace.

During the same [un]prompted conference, Tim de Waal, a security researcher from OpenAI, showcased instances where an AI agent successfully patched two specific vulnerabilities within the firmware of a network access point. However, the execution was not without flaws; in one instance, the AI agent rebuilt a vulnerable function but omitted critical stack canaries present in the original code, effectively disabling a safeguard. For another issue, the generated patch filtered out double-quote characters in attacker inputs but overlooked the possibility of single quotes, which would bypass this fix entirely.

Controlled experiments mirrored these findings at a larger scale. Researchers at 1Password’s Off-by-1 Labs employed AI agents to generate patches for six known vulnerabilities across complex open-source projects, resulting in about half of the 6,080 generated patch candidates leaving at least one attack vector unaddressed. The models sometimes failed to identify identical instances of the same bug scattered throughout the code.

More recently, in an intriguing benchmark test known as P2Patch, two AI agents were pitted against one another—one aiming to produce exploits for 101 real vulnerabilities while the other developed patches for the code until the exploits ceased functioning. Alarmingly, in 41 of the 101 instances, the exploit agent succeeded in breaching the patched programs, often multiple times following subsequent fixes.

Despite these setbacks in AI-generated patches, it is crucial to recognize that human-written patches are not without their failings. A review by the security firm Trail of Bits highlighted that, among the 2,265 vulnerabilities identified in consulting assessments over several years, one in eight human-developed fixes fell short of fully addressing the issues described.

One documented case underlined this point: both a human maintainer and an AI agent were tasked with resolving the same memory-safety flaw in the freenginx web server, yet both solutions ultimately introduced the same crash during the cleanup phase of requests.

As organizations increasingly turn to AI for patching solutions—largely due to the sheer volume of vulnerabilities emerging—this does not imply that AI-generated patches are inherently superior to human-developed ones; instead, the accelerating rate of vulnerability discovery has outpaced human capacity for remediation. The efficiency of AI agents now allows attackers to reverse-engineer and analyze patches at remarkable speeds.

For instance, Anthropic’s Mythos-powered Glasswing program, launched just seven months prior, recorded an astonishing 37,966 findings out of which only a small fraction, 217 of 7,349 identified flaws, had been resolved. This observation highlights the overwhelming number of reports that maintainers must sift through, as not every finding necessarily constitutes a security vulnerability.

In light of this evolving landscape, it has become evident that the speed at which new vulnerabilities are discovered is now outpacing the ability to verify, disclose, and patch them efficiently—an insight expressed by Anthropic in their assessments.

The implications extend beyond open-source projects; even major corporations are affected. Following an engagement with Anthropic’s technologies, Bank of America recruited an additional 1,000 outside contractors to assist in patching efforts, supplementing their existing workforce of around 3,000 staff members. As late as October, CEO Brian Moynihan confirmed that these additional personnel were still actively engaged in this endeavor.

In a related development, by July, Meta began leveraging AI for addressing 40% of its critical vulnerabilities, with nearly half of these patches being deployed in production without alterations. Nonetheless, despite this advancement, the duration of exposure did not decrease significantly; Meta’s software engineer David Molnar indicated that ongoing human review processes played a substantial role in this stagnation.

Molnar articulated the severity of the issue, revealing that the growth of new code—over 100% year over year—was largely facilitated by AI coding agents, illustrating the scale of the data review problem. He emphasized that there is a dire need for organizations to not just automate the process but to consider the entire workflow, advocating a transition from a “human-in-the-loop” approach to a “human-on-the-loop” model.

Considering the escalating threat landscape, the Forum of Incident Response and Security Teams (FIRST) projected that approximately 66,000 CVE assignments would occur for the year. However, as the year progresses, this figure is rapidly approaching 100,000, emphasizing the relentless pace at which vulnerabilities are emerging. Notably, Microsoft’s October Patch Tuesday alone aimed to tackle 1,415 vulnerabilities, a stark increase from the mere 180 identified in October of the previous year.

In summary, the emergence of AI in the cybersecurity landscape serves as a double-edged sword; while it offers the potential for more rapid identification and remediation of vulnerabilities, it simultaneously enables attackers to exploit gaps in patching efforts with unprecedented speed and sophistication. Moving forward, organizations must prioritize not only the detection of vulnerabilities but also engage in more diligent patching strategies that effectively consider the context and complexity of flaws.

Source link

Exit mobile version