Custom-built artificial intelligence (AI) applications are poised to create significant challenges for security teams in the coming years, according to a warning issued by Gartner, a leading research and advisory company. The analyst firm forecasts that by the year 2028, at least half of all enterprise incident response efforts will be centered on managing the repercussions of security issues linked to these increasingly prevalent AI systems.
Christopher Mixter, a Vice President and Analyst at Gartner, emphasized the rapid evolution of AI technology. He highlighted a critical concern: many custom-built AI applications are being deployed without undergoing thorough pre-deployment testing. “These systems are complex, dynamic, and difficult to secure over time,” Mixter stated. The lack of established protocols for addressing AI-related security incidents poses serious challenges, as security teams often find themselves ill-equipped to handle such issues efficiently. Consequently, problems can linger unresolved for longer periods and demand more resources to rectify.
In light of these challenges, Mixter suggested that security teams can mitigate risks by “shifting left,” a strategy that encourages integrating security measures early in the development cycle of AI applications. By implementing adequate controls from the outset, organizations can ensure that security is a foundational aspect of AI deployment rather than an afterthought.
Further amplifying the urgency of these security concerns, Gartner’s predictions indicate the rise of AI-powered security tools within organizations. According to the firm, in just two years, approximately half of all organizations will rely on AI security platforms specifically designed to safeguard the use of third-party AI services and custom-built AI applications. These innovative tools are expected to assist security teams in enforcing acceptable use policies, monitoring user activities, and instituting consistent safety regulations across AI systems. In doing so, these platforms can provide protection against threats like prompt injection and data misuse, which have become increasingly pertinent as AI technology evolves.
Additionally, Gartner noted the emergence of AI-powered “identity visibility and intelligence platforms.” These platforms are becoming essential as organizations strive to improve the detection and resolution of both human and machine identities in their systems. Supporting this assertion, a report from Sysdig published last year revealed that machine identities now outnumber human users by a staggering ratio of 40,000 to one, introducing 7.5 times more risk. A particular concern lies with over-permissioned AI agents, which pose a significant threat to organizational security.
Shifting focus from AI applications, Gartner’s predictions delve into the broader implications of cloud security and organizational governance. The firm projected that by next year, nearly one-third (30%) of organizations will demand “comprehensive sovereignty” over their cloud security controls to mitigate the risks associated with ongoing geopolitical uncertainties. This shifting landscape is driven by local regulatory demands and the ever-evolving geopolitical climate, necessitating that Chief Information Security Officers (CISOs) play an instrumental role in defining these sovereignty requirements.
However, research conducted by UK startup Arqit highlighted that unless handled judiciously, such mandates for data sovereignty could hinder innovation. Notably, 62% of surveyed organizations reported that data sovereignty and privacy concerns are the primary factors delaying AI projects when utilizing public cloud services. Andy Leaver, CEO of Arqit, remarked, “At the same time, one in six organizations inform us that they cannot guarantee sovereign facilities at all, and only 8% can rely on sovereign edge environments today.” Such discrepancies indicate an urgent need for security leaders to establish more robust controls for data management while enhancing their visibility into cryptography.
In response to these challenges, Arqit and other companies are advocating for confidential computing as a viable solution to achieve sovereignty without compromising security. This innovative approach creates secure enclaves at the processor level, allowing for the isolation and protection of data in use, potentially easing the transition to achieving comprehensive sovereignty in cloud environments.
In summary, as custom-built AI applications continue to proliferate, security teams face impending challenges that require early engagement and strategic foresight. The adoption of AI-driven security tools and a focus on data sovereignty will be critical in navigating this rapidly changing landscape. Organizations must adapt to the evolving threat landscape to safeguard their operations effectively.