The Growing Need for AI Security: Addressing Challenges for Organizations
As artificial intelligence (AI) tools become increasingly prevalent in the business landscape, the parallel growth in security measures has proven insufficient. A recent survey, "State of AI: Global Survey 2025" conducted by McKinsey, revealed that an impressive 88% of organizations have integrated AI into at least one of their business functions. Despite this surge in adoption, IBM’s "Cost of a Data Breach Report 2025" highlighted alarming findings: 13% of organizations experienced breaches specifically tied to AI models or applications, with a staggering 97% of these incidents occurring in organizations lacking adequately established AI access controls.
For Chief Information Security Officers (CISOs), this presents a dual challenge. They must develop security measures—often termed "guardrails"—that protect their organizations without stifling the innovation and advantages that AI technologies offer. Internal AI tools such as large language models (LLMs), copilots, assistants, and autonomous agents bring forth unique risks that traditional security frameworks are ill-equipped to handle. Addressing these risks necessitates thorough governance, robust technical controls, and vigilant monitoring.
Establishing Governance as a Priority
Before delving into technical controls, establishing governance is paramount. Organizations are advised to appoint a dedicated individual responsible for overseeing AI initiatives. This person should not only possess the authority to enforce policies but also have the mandate to coordinate efforts across various departments including security, privacy, legal, and business teams.
A comprehensive risk register is essential for tracking the benefits and threats associated with AI. Organizations should define specific AI policies that cover aspects such as acceptable use, data handling, and training requirements. Established frameworks like the National Institute of Standards and Technology (NIST) AI Risk Management Framework and ISO/IEC 42001:2023 can provide structured guidance.
NIST Special Publication 800-221A serves as a practical resource organized around two core functions: governance—encompassing roles, context, benchmarking, policy, and communication—and management, which pertains to risk identification, analysis, prioritization, response, and ongoing monitoring.
Linking AI governance to broader enterprise strategy is vital. When AI-related risks align with business objectives, leadership is more likely to take notice and act decisively.
Designing Robust AI Security Guardrails
Technical guardrails must reflect the distinctive threats associated with internal AI implementations. Specific categories that need addressing include:
-
Data Protection: To prevent sensitive information from infiltrating AI systems, organizations should classify data prior to its introduction to any model or agent. Establishing data loss prevention (DLP) controls on AI interfaces and monitoring for personally identifiable information in both prompts and outputs is crucial.
-
Access and Identity Management: AI agents operate in a unique space straddling tools and users, creating an identity gap that traditional Identity and Access Management (IAM) models fail to encompass adequately. Implementing zero-trust principles for agent permissions can mitigate this issue: granting limited access required for specific tasks and ensuring that permissions expire automatically. Additionally, requiring human approval for critical operations is advised.
-
Prompt and Interaction Security: With prompt injection being a prevalent attack vector for AI systems, it is essential to validate and sanitize all user inputs. This includes separating system prompts from user-provided content and constraining agent actions using allowlists while deploying anomaly detection to identify unusual command sequences.
- Monitoring and Human Oversight: Organizations should log all agent actions and authentication attempts, employing a Security Information and Event Management (SIEM) system to correlate activities across systems. Setting up escalation paths to trigger human review for any anomalous behavior can prevent potential damage.
Extending Guardrails to Development and Supply Chain
Security measures must extend beyond immediate AI implementations to encompass the software development lifecycle (SDLC) and supply chain. Organizations are urged to vet third-party AI models, plugins, and integrations prior to deployment. Historical incidents, such as those involving OpenClaw, illustrate how exposed administrative interfaces, leaked API keys, and the absence of sandboxing can create cascading vulnerabilities across interconnected systems.
AI agents that fetch updates from external sources or adopt third-party skills introduce significant supply chain risks, necessitating the same level of scrutiny applied to traditional software dependencies. Testing models for adversarial inputs, reviewing agent permissions during code audits, and incorporating AI-specific threat modeling into the SDLC are essential practices.
Operationalizing Security Measures
Establishing security guardrails is a continuous effort. Organizations must devise incident response plans tailored to AI-specific scenarios, such as agent compromises and potential data exfiltration incidents via AI interfaces. Notably, incidents involving unauthorized use of unapproved AI tools, commonly referred to as shadow AI, necessitate heightened vigilance. IBM’s report emphasizes the significant costs associated with these shadow incidents, which can add substantial financial burdens to breach management.
Routine AI risk meetings should be scheduled to review the risk register and assess the effectiveness of current controls, allowing for adjustments as threats evolve. Compliance becomes increasingly critical with the introduction of regulations such as the EU AI Act and state-level laws like NYC Local Law 144, which impose mandatory requirements on high-risk AI systems. Organizations should design their security measures to comply with these regulations from the outset rather than treating compliance as an afterthought.
Immediate Steps for CISOs
To secure AI usage in their organizations, CISOs should consider the following actions:
- Appoint a dedicated AI governance lead with clear authority and oversight.
- Develop a risk register encompassing both the advantages and risks associated with AI.
- Classify data that AI systems can access and implement appropriate DLP controls.
- Enforce zero-trust identity principles across all AI agents.
- Conduct audits of third-party AI components to assess supply chain risks.
- Create incident response plans specifically tailored to AI-related scenarios.
- Regularly schedule AI risk reviews that align with broader business objectives.
CISOs should avoid common pitfalls, including treating AI security as a one-time project, granting overly broad permissions to simplify operations, neglecting shadow AI until breaches occur, and postponing governance until regulatory pressures necessitate action.
As the pace of AI adoption accelerates, organizations that proactively secure their AI implementations will position themselves to innovate with confidence, driving future success.
Matthew Smith serves as a fractional Chief Information Security Officer (vCISO) and management consultant, specializing in cybersecurity risk management and the implications of artificial intelligence.