Agentic AI,
Artificial Intelligence & Machine Learning,
Next-Generation Technologies & Secure Development
AI Pilot Projects Are Widespread But Few Agencies Know Who Answers for a Failure
In a rapidly evolving landscape of technology, many federal agencies find themselves at the forefront of experimenting with autonomous artificial intelligence systems. Despite this trend, a recent survey conducted by Booz Allen Hamilton reveals that less than a third of federal technology leaders possess confidence in their agencies’ abilities to deploy these advanced systems securely. The survey illustrates a critical gap between the ambitious pilot projects being undertaken and the perceived trustworthiness of those initiatives.
Findings from the survey indicate that while a significant 51% of respondents confirmed their agencies were actively piloting or testing agentic AI systems, only a mere 7% reported having these systems deployed and operational in a live environment. Additionally, while 22% expressed intentions to deploy agents within the next year, the overall lack of confidence raises several concerns about the implementation of AI technologies within federal frameworks.
A striking statistic from the survey reveals that only 28% of participants felt “extremely” or “very” confident in their agency’s ability to implement agentic AI securely. Intriguingly, half of the surveyed individuals stated they were moderately confident, while the remaining 23% expressed only slight or no confidence at all. These sentiments were captured during an online survey in April, which targeted 105 decision-makers and influencers within the federal government who have a stake in IT and cybersecurity strategy.
The nature of agentic AI systems significantly contributes to the apprehension surrounding their deployment. Unlike generative AI chatbots, which typically respond to queries with limited capabilities, AI agents are designed to autonomously carry out tasks, utilizing software tools and accessing databases with minimal human oversight. This autonomy brings with it a host of security risks, as these agents could exploit valid permissions in unintended ways to achieve their objectives, thus raising alarms about their potential for misuse.
Vibhuti Sinha, the chief product officer at Saviynt, an identity-security firm, elucidates the complications associated with agents. Unlike traditional human or deterministic non-human identities, which can be confined within defined boundaries, agents possess a level of autonomy that complicates security measures. Sinha noted, “You can define a scope, you can define a boundary for your human identities or even deterministic nonhuman identities, and they will always honor or operate within that boundary. That’s not the case with agents because agents are autonomous in nature.”
The survey identified several critical concerns that federal agencies face when considering the implementation of agentic AI. The foremost concern, flagged by 56% of respondents, was the protection of sensitive or classified data. Following closely, half of the participants cited unauthorized agent actions as a major risk, while 37% were worried about adversarial manipulation and prompt injection. Additional risks included the lack of explainability, which could hinder post-incident audits (34%), as well as cascading failures that might arise when automated systems interact (22%).
These findings underscore the challenges of applying conventional identity and access controls to decision-making software capable of independent actions. Sinha emphasized the importance of distinguishing intent from appropriateness, especially in the context of agentic AI: “Intent versus appropriateness are two very different things. With agents, the biggest problem is intent deviation.”
In light of this, federal cybersecurity agencies have begun formulating guidance to tackle the multifaceted challenges posed by agentic AI. In May, the Cybersecurity and Infrastructure Security Agency, along with the National Security Agency and international partners, published security guidance focused on these autonomous systems. Recommendations included restricting agent privileges, monitoring behavior, retaining detailed logs, and instituting mechanisms that allow for the interruption or disabling of agents if necessary.
In a related initiative, the National Institute of Standards and Technology (NIST) has launched an AI Agent Standards Initiative aimed at enhancing agent security, identity management, and interoperability.
The survey also revealed a lack of clarity regarding accountability for agent behavior, with responsibilities being distributed across various organizational layers. Mission or program owners were assigned responsibility by 26% of respondents, while 22% indicated that IT infrastructure owners bore the burden of accountability. Alarmingly, another 22% admitted that their agencies had yet to establish clear guidelines regarding who would be deemed responsible in the event of a security incident or operational failure caused by an agent.
To bolster confidence in agentic AI deployments, federal leaders cited several key strategies. Better monitoring tools for agent behavior were highlighted as a necessity by 56% of respondents. Additionally, 44% believed that having risk mitigation playbooks and best practices would enhance their confidence levels. Furthermore, 42% indicated that a proven track record from previous government applications would serve to solidify trust in these technologies.
Amid these challenges, the survey also addressed the dark side of AI, with many federal leaders expressing concerns about adversaries using AI agents to compromise security. A staggering 79% of participants voiced that they were either extremely or very concerned about the prospect of adversaries leveraging AI to accelerate cyberattacks within the upcoming 12 to 18 months.
The apprehensions extended to specific AI-enabled threats, with 85% of respondents expressing concern over attacks aimed at manipulating, blinding, or corrupting AI systems. Additionally, 84% identified the accelerated exploitation of vulnerabilities through AI tools as a significant risk, and 83% expressed concerns over autonomous attack agents capable of adapting to defenses and persistently operating without human intervention. Amidst these threats, only 36% of respondents held the belief that current AI-powered defensive systems could effectively counteract this rising tide of AI-enabled attacks, while 50% remained uncertain, and 13% outright dismissed the efficacy of such defensive measures.