Why Better AI Models Alone Can’t Establish Reliability in Autonomous Security
In today’s cyber landscape, organizations are increasingly incorporating artificial intelligence (AI) into their security operations. From investigating alerts to identifying vulnerabilities, AI is touted as a transformative technology capable of streamlining operations by prioritizing risks and recommending solutions. However, there is a significant distinction between seeking recommendations from AI and allowing it to autonomously execute those recommendations, a leap that many organizations are currently hesitant to take.
According to a 2026 survey conducted by the Cloud Security Alliance, only 13% of organizations have fully autonomous agentic AI models, revealing a widespread reluctance to trust these systems. Instead, most enterprises choose to deploy AI agents to manage lower-risk tasks while human operators reserve the right to make higher-risk decisions. This cautious approach is often attributed to a lack of trust in AI; however, experts argue that the issue extends beyond trust—it hinges on the complexities of understanding AI behavior and the visibility into its operations.
While it might be easy to assume that the hesitance stems from a general distrust of AI, the reality is that security teams have historically approached technology with a focus on reliability rather than faith. Trust is typically built from a thorough understanding of system operations and the ability to observe and interpret outcomes when something goes awry. As AI transitions from providing insights to executing actions, maintaining human oversight becomes imperative. Many organizations find themselves in a position where they lack enough evidence to fully relinquish decision-making authority to the AI systems they implement.
The Model Is Only One Element
Considerable efforts have been directed towards refining AI models, focusing on aspects such as minimizing hallucinations, enhancing reasoning, and improving transparency about decision-making processes. While these enhancements are crucial, they do not suffice on their own when AI begins to take autonomous actions. Understanding the AI model’s architecture is equally important; organizations need to comprehend what data the model can access, the limitations of its capabilities, and the repercussions of its actions.
As a stark illustration, one AI model might analyze a code repository to identify vulnerabilities, while another could autonomously modify that code. The implications of these activities are drastically different and reliant on the specific architecture surrounding the AI model. Factors such as connectivity to other systems, available tools, permission levels, and the established protocols for autonomous action all contribute to the model’s efficacy and reliability.
Security teams already employ rigorous methodologies in managing personnel, applications, and infrastructure. With AI now becoming a participant in this ecosystem, it is crucial to treat it with the same diligence and oversight as other components within the security framework.
Cost as an Indicator of AI Behavior
Understanding AI behavior also necessitates a consideration of cost. The operational costs associated with AI deployment can vary widely. For example, a routine task that typically generates a few model calls may rapidly escalate to hundreds. If an AI agent becomes "stuck" in a repetitive task, uses a more costly model than necessary, or continues to run past its intended lifecycle, the result is not just budgetary implications but also a deeper operational problem.
Unusual costs can act as red flags, pointing to inefficiencies, unanticipated behaviors, or erroneous processes that might otherwise go unnoticed. While security teams and FinOps (Financial Operations) teams possess distinct responsibilities, AI serves as a bridge, encouraging collaboration by highlighting shared operational signals that provide insights into how AI is functioning in real-world environments.
Gradual Autonomy
There’s a common misconception that human oversight will be phased out entirely as AI systems evolve. However, experts suggest that this perspective may not be beneficial in the long run. If an AI system successfully identifies a vulnerability and proposes a solution, it remains prudent for a human engineer to review the recommendation prior to execution. Over time, as the AI demonstrates reliability, organizations can gradually grant it the autonomy to handle lower-risk cases without human intervention.
This incremental approach mirrors existing automation practices in sensitive operational contexts. Starting with a restricted scope allows organizations to observe outcomes, rectify issues, and incrementally broaden the AI’s independent capabilities.
Human approval will likely remain relevant for certain decisions, especially those with significant consequences, such as modifying production code or revoking access to sensitive systems. Autonomy in AI does not necessitate uniformity; rather, it should be adapted based on the intricacies of each task.
Building Confidence Through Evidence
While the capabilities of AI systems will continue to advance, mere improvement in functionality is insufficient to tackle the trust issue. For security teams, the principles of understanding the operational environment, instituting robust controls, and retaining sufficient evidence for investigation remain crucial.
AI fundamentally alters the nature of the systems being supervised. Organizations are shifting from traditional software that follows pre-laid instructions to dynamic systems capable of interpreting situations, making decisions, and executing actions. Consequently, organizations do not require flawless predictability from AI; they need a comprehensive understanding of its reliable functions, failure points, and scenarios where human oversight remains essential.
As organizations accumulate evidence surrounding AI behavior, the boundaries of autonomy can be refined. Some actions may seamlessly transition into autonomous operations, while complex tasks that involve significant risk will necessitate continued human oversight. This practical, evidence-based pathway to autonomy avoids the pitfalls of premature trust in AI, ensuring that organizations can confidently integrate AI into their security operations while managing the inherent risks and responsibilities.