Recent trends in corporate technology reveal a growing concern over the integration of artificial intelligence (AI) agents within enterprise resource planning (ERP) systems. As companies increase their reliance on AI for finance, procurement, and supply-chain management, a new category of AI risk has emerged, complicating the security landscape.

Gone are the days when security threats primarily arose from unauthorized access through means such as stolen credentials or social engineering attacks. Modern security and IT teams now grapple with the dual challenge of protecting against external attackers while also managing agents that have legitimate access to critical business systems. This complexity elevates the stakes as AI agents can effectively execute harmful actions within these systems without triggering any immediate alarms.

Roland Palmer, Chief Information Security Officer (CISO) and Vice President of Security at JumpCloud, elaborates on this issue, stating, “The category that is genuinely new doesn’t have an attacker in it. Instead, it’s an agent with legitimate access doing what it was told. No breach, every credential valid, every permission granted.” This newly identified category of security risk requires organizations to rethink their security strategies.

A study conducted by ERP-threat detection and compliance vendor Onapsis indicates that AI integration in ERP systems is surging, with 58% of surveyed organizations reporting the adoption of AI applications or agents within the last six months. Furthermore, over 62% acknowledged the use of AI-generated code in their ERP applications, while another 26% plan to implement such technologies by the end of the year 2026.

Despite this rapid implementation of AI technologies, a notable discrepancy exists between adoption and trust. More than 70% of respondents expressed limited or no confidence in AI’s ability to safeguard their critical data. Moreover, nearly 69% confessed doubts about their existing frameworks’ capabilities to detect AI-driven attacks. Alarmingly, around 22% reported security incidents within the previous year where AI was utilized against essential platforms, while an additional 15.2% suspected similar occurrences without definitive proof.

This apprehension is palpable within organizational structures, as evidenced by the fact that nearly 57% of the surveyed cybersecurity leaders indicated that at least one business unit resisted integrating AI into the ERP environment. The primary pushback came from security teams, with 41.4% expressing skepticism, followed by IT departments at 20.7%. Factors such as a lack of confidence in AI security (75%) and concerns over compliance risks (71.6%) were cited as driving forces behind this resistance.

The ERP AI Threat Landscape

Experts in cybersecurity assert that while AI is equipping attackers with new tools to exploit existing vulnerabilities, it also introduces entirely new categories of risk. Juan-Pablo Perez-Etchegoyen, Chief Technology Officer at Onapsis, notes a significant shift in focus among attackers—from targeting operating systems and databases to zeroing in on ERP systems aided by AI. This transition allows hackers to analyze code more efficiently and generate specialized payloads faster.

Eamonn O’Neill, CTO and Co-Founder of the SAP managed-services provider Lemongrass, underscores the dual nature of AI: while it may facilitate attacker methodologies, organizations that operate ERP systems in the cloud can leverage layered defenses, enhancing their security posture. However, he identifies AI-assisted social engineering as a particularly pressing threat, noting that traditional phishing attempts, which are typically easily recognized, can be vastly improved by AI-generated content that appears far more legitimate.

Palmer emphasized that much of the AI-driven threat landscape emerges from attackers enhancing established tactics. “The patterns aren’t changing, the polish and volume are,” he noted. He identifies the manipulation of authorized agents, which can lead to undesirable outcomes even when they operate with valid credentials, as a crucial new risk to consider.

How to Build Trust in Agents

The Onapsis survey also uncovered strategies that organizations believe could enhance their trust in AI agents within ERP systems. Approximately 62% of respondents indicated that robust access management would foster greater trust, while nearly 46% felt that implementing stronger personal data protections would do the same. Additionally, isolating sensitive data in controlled environments, such as sandboxes or digital twins, was suggested by about 37% of respondents as a means of increasing security confidence.

Perez-Etchegoyen insists that design choices made prior to agent deployment can significantly influence the trustworthiness of these systems. Key measures include ensuring that agents operate under distinct identities with minimum permissions necessary for their tasks—a principle aligned with zero-trust and least-privilege frameworks that can effectively limit the potential fallout in the case of a breach.

O’Neill concurs, urging organizations to treat agent identities as akin to human identities, using existing access management frameworks to enforce segregation of duties and the distinction between reading and modifying data. Before granting any modification rights, organizations should rigorously test permissions and assess potential downstream effects to mitigate risks. Without proper confirmation, he warns, “don’t switch it on.”

Palmer adds that his team treats agents like new employees, granting them identities and limited permissions, gradually expanding access contingent upon their proven reliability. He stresses the importance of human oversight, insisting that accountability for agent actions should remain with actual human staff.

Gate the Feature Not the Vendor

In light of the ongoing integration of AI into ERP systems, it is imperative that organizations maintain rigorous vetting and monitoring processes. For Palmer, this involves a set of crucial questions that need to be posed regarding each AI feature included in their systems: Can it be turned off? What identity does it act as? What actions does it log, and can access be separated from the user’s privileges?

He emphasizes the importance of maintaining a cautious approach, suggesting that “immature answers are workable if we plan for the immaturity using tactics like off-by-default and scoped rollout.” Vendors that cannot provide satisfactory answers should not see their features deployed. “We gate the feature, not the vendor,” he advises, recognizing the challenge of rejecting ERP systems while advocating for a more cautious approach to AI modules.

For organizations developing their own AI platforms, Perez-Etchegoyen recommends adhering to traditional change management protocols, including thorough security testing and code analysis, to prevent vulnerabilities within business workflows. O’Neill echoes this sentiment, stating that no agent should be deployed without a comprehensive review of its access through a recognized governance, risk, and compliance process. This becomes even more crucial in scenarios where an agent possesses the capability to write to an ERP system.

“Almost like you would with a person,” he concludes, “I would not let a person with a role that hasn’t been properly clarified access to an ERP system.”