Industry analysts predict that organizations will intensify their efforts to leverage generative artificial intelligence (GenAI) and large language models (LLMs) across various use cases in the coming year. These use cases span from customer support, fraud detection, content creation, data analytics, knowledge management, to software development. A recent survey conducted by Centient on behalf of OutSystems revealed that 81% of IT professionals are currently utilizing GenAI to assist with coding and software development, with 74% planning to develop 10 or more apps in the next 12 months using AI-powered development approaches.
While the potential benefits of using AI in these scenarios are significant in terms of efficiency and productivity gains, they also bring about new challenges surrounding privacy, governance, and security. Experts in the industry are highlighting six key security issues that IT and security leaders need to be mindful of within the next year.
The first issue centers around the mainstream adoption of AI-based coding assistants such as GitHub Copilot, Amazon CodeWhisperer, and OpenAI Codex. While these tools promise improvements in developer productivity, reduced errors, and faster development times, they also present security risks such as vulnerable code, data exposure, and propagation of insecure coding practices.
Derek Holt, CEO of Digital.ai, emphasizes the need for enterprises using AI coding assistants to implement thorough security measures like Dynamic Application Security Testing (DAST) and Static Application Security Testing (SAST) to identify vulnerabilities and ensure secure code practices.
Another concern highlighted by experts is the convergence of DevSecOps, DataOps, and ModelOps into a more comprehensive management approach known as xOps. This shift is driven by the integration of AI capabilities into software development, which blurs the lines between traditional apps and dynamically generated responses from LLMs and GenAI apps. This trend will require operations, support, and QA teams to adopt xOps practices to manage the complexities of AI-enabled software effectively.
The unauthorized use of GenAI tools, commonly referred to as Shadow AI, poses a significant security challenge for organizations. The proliferation of AI chatbots within workplaces has raised concerns about data exposure and compliance issues. Experts anticipate a rise in the unsanctioned use of AI tools, necessitating chief information officers and chief information security officers to enhance their capabilities for detecting and managing Shadow AI.
AI’s role in cybersecurity is expected to augment human skills rather than replace them entirely. While AI excels at processing vast amounts of threat data and automating routine tasks, human expertise remains crucial for identifying and responding to sophisticated attacks. Organizations will need to strike a balance between AI-driven automation and human intuition to effectively combat evolving threats.
Furthermore, attackers are likely to leverage AI tools to exploit vulnerabilities in open-source software and execute zero-day attacks. The use of AI-enabled ransomware and AI-driven research to identify vulnerabilities pose significant cybersecurity risks that organizations must address proactively.
Maintaining trust in AI systems remains a complex challenge, with many senior managers and customers expressing doubts about AI’s reliability. Implementing robust verification systems and incorporating human oversight are essential strategies to mitigate risks and ensure the ethical use of AI technologies.
In conclusion, the acceleration of AI adoption across various industries brings new opportunities and risks that organizations must navigate carefully. By addressing the identified security issues and leveraging AI alongside human expertise, enterprises can harness the full potential of AI technologies while safeguarding against potential threats and vulnerabilities.