Autonomous Systems Require Defined Identities, Controlled Access, and Human Oversight
In the current business landscape, a growing number of enterprises are granting artificial intelligence (AI) agents the authority to act on their behalf. However, the systems necessary for identity and governance surrounding these AI agents are not evolving at the same pace. This discrepancy has raised important questions about how to effectively manage these autonomous entities as they gain more control over systems, permissions, and workflows.
A recent survey conducted by JumpCloud, a cloud-based directory and IT management platform vendor, has revealed significant insights into the use of AI agents in modern enterprises. The survey uncovered that a noteworthy 55% of organizations are either currently using or testing AI agents capable of modifying systems and records. Yet alarmingly, of those organizations employing such agents, 59% have not fully integrated their human identity and access management (IAM) policies for these non-human entities.
Andras Cser, Vice President and Principal Analyst at Forrester, articulated the nature of AI agents as "smarter, less deterministic APIs." He highlighted that the combination of scale and autonomy at the enterprise level creates an amplifying effect. Thus, it becomes possible for organizations to possess a greater number of agent identities than human users, each often acting with more independence than traditional machines could manage.
For Chief Information Officers (CIOs) and Chief Information Security Officers (CISOs), the pressing challenge lies in incorporating these agents into their pre-existing enterprise identity frameworks. A failure to do so may result in agents undertaking actions that deviate from the organization’s intent. Cser emphasized that AI agents should be managed within the current identity and access management frameworks, rather than through a completely separate system. As new identity types, AI agents still require oversight in line with existing policies that govern human users and other machine identities.
The process of effectively managing these agents begins with a thorough discovery phase. Organizations are urged to compile an inventory of all operational agents, create an approved pool of agent types and providers, and establish a review process for the agent roster as their capabilities and associated risks evolve. Each agent must have its own distinct identity and credentials to ensure accountability while also maintaining the relationship with the human user or system that authorized its actions.
Cser stressed that it is critical for an accountable authority to be identified and involved at the end of every agent’s decision-making process. He expressed concern over the scenario where agents operate without proper oversight, noting, "You cannot just say, ‘We launched a bunch of agents from a script, and it did whatever it did.’"
In line with this, the National Institute of Standards and Technology (NIST) also supports the classification of agents as "first-class entities." In a recent blog post, NIST researchers Bill Fisher and Ryan Galluzzo argued for the necessity of unique identifiers, credentials, and entitlements tied directly to the user or system responsible for the agents’ actions. Furthermore, they cautioned against the use of shared human credentials or static API keys, advocating instead for mechanisms like OAuth 2.0 and SPIFFE to issue short-lived, tightly scoped credentials.
Determining an agent’s identity is merely the starting point. Organizations also need to assess whether these agents are permitted to execute specific actions and when these actions can take place. Cser delineated the need to evaluate "three identities and three contexts": the human user, the agent, and the resource targeted for action. For instance, when assessing a procurement agent, various factors such as the employee’s role, geographical location, budget, and the agent’s permissions must be taken into account.
JumpCloud’s survey revealed disconcerting trends in organizations’ handling of agent actions. It indicated that 46% of organizations permit high-risk actions to proceed automatically, with subsequent log reviews. Meanwhile, 29% allow actions to occur with limited or no review, while a mere 18% mandate prior human approval. However, the necessity for prior approval will not remain uniform across all actions. NIST raised concerns about "consent fatigue," where users become conditioned to approve requests without due diligence.
When agents do take action, it is imperative for organizations to retain comprehensive audit trails documenting the rationale behind each decisional step. According to Cser, these records should detail what agent was involved, the runtime context, the delegating human, timestamps, actions, credentials, resource identifiers, and reason codes justifying the agent’s chosen path.
In conclusion, as CIOs navigate the complexities of integrating AI agents into their operational frameworks, they need to approach the task methodically. They are advised to start with a detailed discovery phase, establish a defined registry, designate unique identities with accountable owners for each agent, and link access decisions to the broader business context. Importantly, the justification for deploying an agent must outweigh the governance costs. Cser insightfully remarked, "Not everything requires an AI agent," thus emphasizing the need for critical evaluation before integration.